Updated changelog for v0.32

Scintilla doesn't strictly follow SetEOLMode setting and accepts both 0d
and 0d0a (and possibly 0a too) as line endings on Windows. Unfortunately,
the standard Lua interpreter does the right thing and only recognizes 0d0a
as the valid marker. This means line numbers are going to be different in
two cases, which creates a problem for breakpoints during debugging.

CHANGELOG.md

@@ -0,0 +1,235 @@
+# ZeroBrane Studio Changelog
+
+## v0.32 (Sep 03 2012)
+
+### Highlights
+  - Added **Unicode support** for file encoding and file paths on Windows (fixes #30).
+  - Added **Moai integration and debugging** (including debugging of Moai threads and callbacks).
+  - Added refresh of Stack and Watch windows after executing a statement in remote shell.
+  - Added **display of complex values** on multiple lines in shell with '='.
+  - Added calltip on mouseover for functions during editing and for variables/expressions during debugging.
+  - Added configuration options to set paths to lua and love2d executables.
+  - Added support for **coroutine debugging** with stepping through coroutine.resume/.yield calls.
+  - Updated wx.dll to wxlua 2.8.12.2 and wxwidgets 2.8.12.
+  - Signed zbstudio app and executable to avoid issues with files not being saved without admin privileges and to remove warning about 'unknown publisher' on windows (fixes #25).
+    
+### Improvements
+  - Added calltip on mouseover for functions during editing and for variables/expressions during debugging.
+  - Added an IO filter to fix an issue with 0d0d0a line endings on Windows.
+  - Added support for debugging moai callbacks (upgraded to mobdebug v0.489).
+  - Added refresh of Stack and Watch windows to show updated values after executing a statement in remote shell.
+  - Added display of complex values on multiple lines in shell with '='.
+  - Added rockspec to the list of extensions for lua (fixes #37).
+  - Added a check to avoid evaluating keywords in tooltip.
+  - Added current interpreter to the status bar; adding closing debugger when the interpreter is changed.
+  - Added aborting scratchpad processing when an interpreter can't start or report a fatal error.
+  - Added support for unicode path files on Windows (fixes #30).
+  - Added an option to set path to lua executable.
+  - Added error handler to trap and display debugger errors.
+  - Added search in PATH for love2d executable.
+  - Added a workaround for GetExecutablePath() reporting 'wx.dll' instead of a proper exe name with wxlua 2.8.12.2 on Windows.
+  - Added reporting of function name of the form a.b and a:b in static analysis (fixes #27).
+  - Added ability for user to keep their settings file in their home directory.
+  - Added per user settings file. Users can now move their settings file to ~/.zbs/user.lua.
+  - Added ignoring Cmd-key combinations on Mac as this should be handled by wxwidgets, but is not (fixes #19).
+  - Added support for coroutine debugging with stepping through coroutine.resume/.yield calls.
+  - Changed reporting of program execution time from CPU time to user time.
+  - Changed the call to unhide windows to the async version (ShowWindowAsync) to avoid blocking the IDE when the application doesn't respond.
+  - Upgraded to wxlua 2.8.12.2 (wxwidgets 2.8.12; unicode version); added lua51.dll proxy (fixes #10 and #7).
+  - Updated love2d interpreter to use the project folder to check for main.lua.
+  - Updated test module to use stringified values for comparison.
+  - Updated status bar style to make it consistent across platforms.
+  - Removed .bak files from being replaced in when backup copies are saved.
+  - Removed explicit path conversions and comparisons.
+  - Refactored LUA_PATH/CPATH processing to set it for all interpreters.
+  - Signed zbstudio app and executable to avoid issues with files not being saved without admin privileges and to remove warning about 'unknown publisher' on windows (fixes #25).
+
+### Incompatibilities
+  - Reassigned hotkeys in the Project menu to minimize conflicts on Mac (reassigned Shift-F12 and F11).
+
+### Fixes
+  - Fixed an issue with double click on analylsis results being out-of-sync when the editor switched to another file (fixes #38)
+  - Fixed an issue with debugger not activating files with relative path information.
+  - Fixed 'break' command to work after coming from debugger calls (like on()).
+  - Fixed an issue with highlighting selected item in the project tree.
+  - Fixed evaluation of foo:bar in tooltip (now evaluates as foo.bar).
+  - Fixed debugger termination after internal errors.
+  - Fixed activating current file in the project tree on Mac (closes #29).
+  - Fixed running scripts with single quotes in path names.
+  - Fixed an issue with Run/Debug commands when IDE path includes exclamation mark ('!').
+  - Fixed an issue with the app not starting on those systems that don't have HOME environment variable; fixes #28.
+  - Fixed an issue with showing/hiding GUI windows that was occasionally causing a runtime error when the window disappears before it is manipulated.
+  - Fixed returning proper name for unsaved files in reporting compilation and static analysis results; moved default names to ide.config (fixes #26).
+  - Fixed pasting text into the Find dialog and project path box on Mac (fixes #22).
+  - Fixed handling of dashes in paths (upgraded to mobdebug 0.479).
+  - Reorganized handling of automcomplete event (to use AddPendingEvent instead of PostEvent) to avoid runtime application error.
+
+## v0.31 (Jul 14 2012)
+
+### Highlights
+  - Added **scratchpad support for love2d**.
+  - Added tooltip to display variable/expression values during debugging.
+  - Added **MacOS support**.
+
+### Improvements
+  - Added handling of balanced brackets in markup links.
+  - Added unit test module.
+  - Added reporting the number of traced lines during debugging.
+  - Added setting of PATH and CPATH to find proper libs on windows and mac os platforms.
+  - Added scratchpad support for love2d.
+  - Added reset of 'modified' status to keep tab names and their config settings correct upon exit.
+  - Added window title update and filetree refresh after SaveAs command.
+  - Added tooltip to display variable/expression values during debugging.
+  - Made 'View Stack Window' and 'View Watch Window' refresh window content if it's already shown.
+  - Removed setting the editor font in the config as the default font is different on different platforms.
+  - Removed extension from the template to match folders to make it more portable.
+  - Reorganized handling of font configuration and added font config for filetree (with a different size default on MacOS).
+  - Updated matching logic for function definitions to allow for a.b.c() definitions (fixes #17).
+
+### Fixes
+  - Fixed markup styling and file tree drawing on MacOS.
+  - Fixed detecting executable name in commands with spaces.
+  - Fixed incorrect folders reported in the file tree when no project directory is set and a file is open.
+  - Fixed incorrect filename reported in compile errors when the file is not saved.
+  - Fixed refresh of filetree on MacOS to get it displayed correctly when the app is started.
+  - Fixed an error thrown when a window with debugging is closed before the application being debugged is terminated.
+  - Fixed incorrect storing of settings for editor tabs with the same text (filename). This was causing only one tab displayed for multiple StyledText controls with interesting effects.
+  - Fixed an issue with launching a process when its output is not redirected to the IDE (fixes #16). 
+  - Fixed console to evaluate 'function a() ... end' without errors.
+  - Fixed a compilation error caused by shebang in scripts.
+  - Fixed an issue with love2d path with spaces.
+  - Corrected resetting of project directory when it's already set and doesn't need to be changed.
+  - Added checks around ShowFullScreen() calls to avoid failures on those systems that don't provide it (linux/GTK).
+  - Added check for debugger calls to avoid errors when debugger is not loaded.
+  - Updated matching of links to make them less greedy (to avoid capturing link terminators).
+  - Upgraded deprecated constants and logic for compatibility with wxwidgets 2.9.x.
+  - Reset project directory if the current one doesn't exist.
+  - Removed styling of function calls and capturing definitions in strings and comments (fixed #18).
+  - Removed setting focus to the Output window when output is processed as it interfered with Run as Scratchpad.
+
+## v0.30 (Jun 27 2012)
+
+### Highlights
+  - Added **love2d support**.
+  - Added auto complete for love2d API.
+  - Added support for debugging processes running under LuaJIT.
+  - Added display of **hierarchical data in Stack window**.
+  - Added **pretty printing in Watch and Console** (local and remote) windows and handling of multiple results in Console.
+  - Added **Stack window to display stack information** and local/upvalue values for each stack frame.
+  - Added ability to **interact with scripts** by allowing text to be entered in the 'Output' window.
+
+### Improvements
+  - Added love2d support.
+  - Added auto complete for love2d API.
+  - Added support for debugging processes running under LuaJIT.
+  - Added display of hierarchical data in Stack window.
+  - Added execution time and updated messages in the Output window to be more consistent.
+  - Added displaying 'nil' values in local console when no result is returned by an expression.
+  - Added a check to refuse starting a new debugging session if there is one in progress already.
+  - Added handling of tail calls in the Stack window.
+  - Added pretty printing in Watch and Console (local and remote) windows and handling of multiple results in Console.
+  - Added Stack window to display stack information and local/upvalue values for each stack frame.
+  - Added ability to set font encoding in the config.
+  - Added restoring cursor position when a modified file is reloaded in the editor.
+  - Added ability to interact with scripts by allowing text to be entered in the 'Output' window.
+  - Improved logic in love2d integration to distinguish Debug and Run commands (closes #13).
+  - Improved reporting in static analysis for functions and global variables.
+  - Updated menus to avoid conflicts with MacOS shortcuts.
+  - Updated logic creating menubar to make it work correctly on MacOS with special Help/About items.
+  - Updated path handling to better detect how the app is started and to avoid loading dlls on non-windows platforms.
+  - Updated logic for detecting hostname (used in the debugger) to make sure it is resolvable.
+  - Changed order of lualibs/ and bin/ directories in package.path and package.cpath to load included modules first.
+  - Removed extensions from launch commands and updated display logic in the Output window.
+
+### Fixes
+  - Fixed aborting running/debugged programs on MacOS by adding MAKE_GROUP_LEADER option to wxExecute.
+  - Fixed an issue in the logic for setting breakpoints, which ignored breakpoints in luxinia2 debug sessions.
+  - Fixed logic in the local/remote console that returned incorrect error message on executing code like '%s':format(1).
+  - Fixed IDs for Project menu items to allow them to be removed from the menu if needed.
+  - Fixed an issue with remote application not terminating when IDE is closed while debugging is in progress.
+  - Fixed refreshing a modified file when the editor is set to read-only mode.
+  - Fixed saving/restoring configuration of 'Output'/'Console' tabs when IDE is closed while debugging is in progress.
+  - Fixed removing variable name in Watch window after escaping editing.
+  - Fixed #9 as it had incorrect logic in one of UTF filters.
+  - Fixed edit menu shortcuts to work in the 'Output' window (when allowed).
+  - Fixed reporting of processes that failed to start after 'Run' or 'Debug' commands.
+  - Fixed executable path matching to work on systems that don't have file extensions.
+  - Fixed #3 'unused parameter...' check not to fail on anonymous functions that are part of an expression.
+  - Moved processing of `user.lua` to a later phase after tools and specs are already loaded to allow modification of IDE configuration from `user.lua`. Closes #5.
+  - Added checks to prevent text modification in 'Output' and 'Console' windows. Fixes #8.
+  - Disabled 'Run as Scratchpad' if there is no debugger registered capable of running it.
+  - Disabled Stack and Watch updates when scratchpad is active as they interfere with application execution.
+
+## v0.29 (May 31 2012)
+
+### Highlights
+  - Added **scratchpad** (running live) functionality.
+  - Added **code analyzer** based on lua-inspect.
+  - Updated **comment styling** to follow markdown syntax.
+
+### Improvements
+  - Added scratchpad (running live) functionality.
+  - Added code analyzer based on lua-inspect.
+  - Added Ctrl(-Shift)-TAB navigation between tabs in the editor.
+  - Added navigation between editor tabs using Ctrl-PgUp and Ctrl-PgDn.
+  - Added reporting of assignment to global variables in the code analyzer.
+  - Added ability to turn external processes that connect to debugger into a scratchpad.
+  - Added exit from full screen mode using ESC key.
+  - Added reporting of compilation errors during debugging sessions.
+  - Added handling of more errors in the shell to allow calculations like '(1+2)' to be executed correctly.
+  - Added moving focus back to the notebook after unhiding/activating a wx window.
+  - Added missing mime/code.dll and reorganized socket module files (socket.*) to load correctly with require.
+  - Added stopping the debugger when a debugged program exits.
+  - Added to static analysis reporting of unused parameters in functions.
+  - Disabled warning in static analysis about unused 'self' in methods.
+  - Removed 'error during pre-compilation' message from compile errors.
+  - Updated comment styling to follow markdown syntax.
+
+### Fixes
+  - Fixed handling of scripts with comments in the remote shell.
+  - Fixed an issue with Analyze process when the analyzed script has compilation errors.
+  - Fixed an issue with scratchpad being on after Save dialog is canceled.
+  - Fixed about screen.
+
+## v0.28 (Mar 21 2012)
+
+### Highlights
+  - Added full screen mode.
+
+### Improvements
+  - Added option to activate output/console when Run/Debug/Compile commands are executed.
+  - Added full screen mode.
+  - Added killing a running process on IDE exit.
+  - Added killing a running process with Shift-F12.
+  - Disabled buffering of the output for scripts run from IDE.
+
+### Fixes
+  - Fixed 'Trace' command to continue working when a debugged file is not activated.
+  - Fixed an issue with saving a file when no project directory is set.
+  - Fixed missing semicolon in lualibs path; added path for debugger to search under lualibs.
+  - Fixed an issue with a missing path separator, which prevented debugging from executing step commands in some cases.
+  - Fixed missing slash on SaveAs by enforcing trailing slash for the project path.
+
+## v0.27 (Feb 14 2012)
+
+### Highlights
+  - Added markup formatting in the comments.
+
+### Improvements
+  - Added markup formatting in the comments.
+  - Added Debug and Run methods to simulate menu commands.
+  - Added setting a project folder on initial start.
+  - Added style processing for font name, font size, visibility and hotspot attributes.
+  - Added setting the current project directory for the shell to allow 'require' commands to work with local modules.
+  - Updated markup processing with run and debug commands, http link processing, and opening local files in a new window.
+  - Enforced visibility for shell prompt.
+
+### Fixes
+  - Fixed activation of a correct tab when one of the editor tabs is closed.
+  - Fixed an issue with file activation from a debugger.
+  - Fixed the issue of ClosePage method being called with two different parameters.
+  - Fixed the issue of the project dir being returned with two trailing slashes.
+  - Fixed an issue with activating the currenly edited file in the file tree.
+  - Wrapped DragAcceptFiles into a protected call to make it not fail on MacOS (compiled with wxwidgets 2.8.12).
+
+## v0.26 (Jan 18 2012)

LICENSE

@@ -2,7 +2,7 @@
 
 ZeroBrane Studio sources are released under the MIT License
 
-Copyright (c) 2011 Paul Kulchenko (paul@kulchenko.com)
+Copyright (c) 2011-2012 Paul Kulchenko (paul@kulchenko.com)
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal
@@ -26,9 +26,9 @@ THE SOFTWARE.
 
 Estrela Editor sources are released under the MIT License
 
-Copyright (c) 2008-2011
+Copyright (c) 2008-2012
   Luxinia DevTeam:
-  Eike Decker & Christoph Kubisch
+  Christoph Kubisch & Eike Decker
   info at luxinia.de 
 
 Permission is hereby granted, free of charge, to any person obtaining a copy

README.md

@@ -1,8 +1,10 @@
+# Project Description
+
 A simple and extensible Lua IDE and debugger. It supports multiple file
 formats, "api" for autocompletion and tooltips, and custom command-line
 tools. Its main focus is extensibility for target applications using Lua.
 
---[[ FEATURES ]]-----------------------------------------------------------
+## Features
 
 * Written in Lua, so easily customizable
 * Automatically loads several 'plugin' like classes
@@ -23,14 +25,27 @@ tools. Its main focus is extensibility for target applications using Lua.
 * Console to directly test code snippets with local and remote execution
 * Integrated debugger (with support for local and remote debugging)
 
---[[ INSTALLATION ]]-------------------------------------------------------
+## Frontends
 
-git clone git://github.com/pkulchenko/ZeroBraneStudio.git zbstudio
+There is currently two front-ends using the same editor engine. The original
+one is `Estrela`, which has a focus on 3d graphics related usage of Lua,
+especially in combination with the luxinia engine or luxinia2 framework.
+The second front-end is `ZeroBrane Studio` (zbstudio) which has a focus
+on using Lua in education, mobile development, and robotics.
+
+Both are part of the standard distribution.
+
+## Installation
+
+```bash
+$ git clone git://github.com/pkulchenko/ZeroBraneStudio.git zbstudio
   or
-git clone git://estrelaeditor.git.sourceforge.net/gitroot/estrelaeditor/estrelaeditor estrelaeditor
+$ git clone git://estrelaeditor.git.sourceforge.net/gitroot/estrelaeditor/estrelaeditor estrelaeditor
+```
 
---[[ USAGE ]]--------------------------------------------------------------
+## Usage
 
+```
 Open File(s):
   <exe> <filename> [<filename>...]
   any non-option will be treated as filename
@@ -38,17 +53,18 @@ Open File(s):
 Overriding Config:
   <exe> [...] -cfg "<luacode overriding config>" [...]
   e.g.: zbstudio.exe -cfg "singleinstance=false;" somefile.lua
+```
 
---[[ AUTHOR ]]-------------------------------------------------------------
+## Author
 
-Estrela Editor
+### Estrela Editor
 
-  Luxinia DevTeam: Eike Decker & Christoph Kubisch (info at luxinia.de)
+  **Luxinia Dev:** Christoph Kubisch (crazybutcher@luxinia.de)
 
-ZeroBrane Studio and MobDebug
+### ZeroBrane Studio and MobDebug
 
-  ZeroBrane LLC: Paul Kulchenko (paul@kulchenko.com)
+  **ZeroBrane LLC:** Paul Kulchenko (paul@kulchenko.com)
 
---[[ LICENSE ]]------------------------------------------------------------
+## License
 
-See LICENSE file
+See LICENSE file.

api/glsl/std.lua

@@ -71,13 +71,14 @@ ldexp = fn "build floating point number from x and the corresponding integral ex
 packUnorm2x16 = fn "Converts each comp. of v into 16-bit ints, packs results into the returned 32-bit uint. - (uint)(vec2 v)",
 packUnorm4x8 = fn "Converts each comp. of v into 8-bit ints, packs results into the returned 32-bit uint. - (uint)(vec4 v)",
 packSnorm4x8 = fn "Converts each comp. of v into 8-bit ints, packs results into the returned 32-bit uint. - (uint)(vec4 v)",
+packDouble2x32 = fn "Packs components of v into a 64-bit value and returns a double-prec value. - (double)(uvec2 v)",
+packHalf2x16 = fn "Converts each comp. of v into 16-bit half float, packs results into the returned 32-bit uint. - (uint)(vec2 v)",
 
 unpackUnorm2x16 = fn "Unpacks 32-bit p into two 16-bit uints and converts them to normalized float. - (vec2)(uint p)",
 unpackUnorm4x8 = fn "Unpacks 32-bit p into four 8-bit uints and converts them to normalized float. - (vec4)(uint p)",
 unpackSnorm4x8 = fn "Unpacks 32-bit p into four 8-bit uints and converts them to normalized float. - (vec4)(uint p)",
-
-packDouble2x32 = fn "Packs components of v into a 64-bit value and returns a double-prec value. - (double)(uvec2 v)",
 unpackDouble2x32 = fn "Returns a 2 component vector representation of v. - (uvec2)(double v)",
+unpackHalf2x16 = fn "Interprets p as two 16-bit half floats and returns them as vector. - (vec2)(uint p)",
 
 length = fn "return scalar Euclidean length of a vector. - (type)(vecN)",
 distance = fn "return the Euclidean distance between two points. - (vecN)(vecN a, b)",
@@ -135,6 +136,11 @@ EndPrimitive = fn "Completes current output primitive and starts a new one. - ()
 barrier = fn "Synchronizes across shader invocations. - ()()",
 
 memoryBarrier = fn "control ordering of memory transactions issued by shader thread. - ()()",
+memoryBarrierAtomicCounter = fn "control ordering of memory transactions issued by shader thread. - ()()",
+memoryBarrierShared = fn "control ordering of memory transactions issued by shader thread. - ()()",
+memoryBarrierBuffer = fn "control ordering of memory transactions issued by shader thread. - ()()",
+memoryBarrierImage = fn "control ordering of memory transactions issued by shader thread. - ()()",
+groupMemoryBarrier = fn "control ordering of memory transactions issued by shader thread. - ()()",
 imageAtomicAdd = fn "performs atomic operation on individual texels returns new value. - (uint)(imageN, intN coord, [int sample], uint data)",
 imageAtomicMin = fn "performs atomic operation on individual texels returns new value. - (uint)(imageN, intN coord, [int sample], uint data)",
 imageAtomicMax = fn "performs atomic operation on individual texels returns new value. - (uint)(imageN, intN coord, [int sample], uint data)",
@@ -167,41 +173,51 @@ texelFetchOffset = fn "integer coordinate lookup for a single texel with offset.
 }
 
 local keyw = 
-[[int uint half float bool double
-vec2 vec3 vec4 dvec2 dvec3 dvec4
-ivec2 ivec3 ivec4 uvec2 uvec3 uvec4 bvec2 bvec3 bvec4
-mat2 mat3 mat4 mat2x2 mat3x3 mat4x4 mat2x3 mat3x2 mat4x2 mat2x4 mat4x3 mat3x4
-dmat2 dmat3 dmat4 dmat2x2 dmat3x3 dmat4x4 dmat2x3 dmat3x2 dmat4x2 dmat2x4 dmat4x3 dmat3x4
-struct typedef void
-usampler1D usampler2D usampler3D usampler2DRect usamplerCube isampler1DArray usampler2DARRAY usamplerCubeArray usampler2DMS usampler2DMSArray
-isampler1D isampler2D isampler3D isampler2DRect isamplerCube isampler1DArray isampler2DARRAY isamplerCubeArray isampler2DMS isampler2DMSArray
-sampler1D sampler2D sampler3D sampler2DRect samplerCube sampler1DArray sampler2DArray samplerCubeArray sampler2DMS sampler2DMSArray
-sampler1DShadow sampler2DShadow sampler2DRectShadow sampler1DArrayShadow sampler2DArrayShadow samplerCubeArrayShadow
-usamplerBuffer isamplerBuffer samplerBuffer
-in out inout uniform const centroid sample attribute varying patch
-return switch case for do while if else break continue
-layout location vertices line_strip triangle_strip max_vertices stream
-triangles quads equal_spacing isolines fractional_even_spacing
-fractional_odd_spacing cw ccw point_mode lines_adjacency triangles_adjacency
-invocations
-origin_upper_left pixel_center_integer
-smooth flat noperspective highp mediump lowp shared packed std140 row_major column_major
-gl_FrontColor gl_BackColor gl_FrontSecondaryColor gl_BackSecondaryColor gl_Color gl_SecondaryColor
-subroutine gl_Position
-gl_VertexID gl_InstanceID gl_Normal gl_Vertex gl_MultiTexCoord0 gl_MultiTexCoord1
-gl_MultiTexCoord2 gl_MultiTexCoord3 gl_MultiTexCoord4 gl_MultiTexCoord5 gl_MultiTexCoord6
-gl_MultiTexCoord7 gl_FogCoord gl_PointSize gl_ClipDistance
-gl_TexCoord gl_FogFragCoord gl_ClipVertex gl_in
-gl_PatchVerticesIn
-gl_PrimitiveID gl_InvocationID gl_TessLevelOuter gl_TessLevelInner gl_TessCoord
-gl_InvocationID gl_PrimitiveIDIn gl_Layer gl_ViewportIndex gl_FrontFacing
-gl_PointCoord gl_SampleID gl_SamplePosition gl_FragColor
-gl_FragData gl_FragDepth gl_SampleMask
+[[  int uint half float bool double atomic_uint binding offset
+    vec2 vec3 vec4 dvec2 dvec3 dvec4
+    ivec2 ivec3 ivec4 uvec2 uvec3 uvec4 bvec2 bvec3 bvec4
+    mat2 mat3 mat4 mat2x2 mat3x3 mat4x4 mat2x3 mat3x2 mat4x2 mat2x4 mat4x3 mat3x4
+    dmat2 dmat3 dmat4 dmat2x2 dmat3x3 dmat4x4 dmat2x3 dmat3x2 dmat4x2 dmat2x4 dmat4x3 dmat3x4
+    struct typedef void
+    usampler1D usampler2D usampler3D usampler2DRect usamplerCube isampler1DArray usampler2DARRAY usamplerCubeArray usampler2DMS usampler2DMSArray
+    isampler1D isampler2D isampler3D isampler2DRect isamplerCube isampler1DArray isampler2DARRAY isamplerCubeArray isampler2DMS isampler2DMSArray
+    sampler1D sampler2D sampler3D sampler2DRect samplerCube sampler1DArray sampler2DArray samplerCubeArray sampler2DMS sampler2DMSArray
+    sampler1DShadow sampler2DShadow sampler2DRectShadow sampler1DArrayShadow sampler2DArrayShadow samplerCubeArrayShadow
+    usamplerBuffer isamplerBuffer samplerBuffer samplerRenderbuffer isamplerRenderbuffer usamplerRenderbuffer
+    in out inout uniform const centroid sample attribute varying patch index true false
+    return switch case for do while if else break continue main inline
+    layout location vertices line_strip triangle_strip max_vertices stream
+    triangles quads equal_spacing isolines fractional_even_spacing lines points
+    fractional_odd_spacing cw ccw point_mode lines_adjacency triangles_adjacency
+    invocations
+    origin_upper_left pixel_center_integer depth_greater depth_greater depth_greater depth_unchanged
+    smooth flat noperspective highp mediump lowp shared packed std140 std430 row_major column_major buffer
+    gl_FrontColor gl_BackColor gl_FrontSecondaryColor gl_BackSecondaryColor gl_Color gl_SecondaryColor
+    subroutine gl_Position gl_FragCoord
+    gl_VertexID gl_InstanceID gl_Normal gl_Vertex gl_MultiTexCoord0 gl_MultiTexCoord1
+    gl_MultiTexCoord2 gl_MultiTexCoord3 gl_MultiTexCoord4 gl_MultiTexCoord5 gl_MultiTexCoord6
+    gl_MultiTexCoord7 gl_FogCoord gl_PointSize gl_ClipDistance
+    gl_TexCoord gl_FogFragCoord gl_ClipVertex gl_in
+    gl_PatchVerticesIn
+    gl_PrimitiveID gl_InvocationID gl_TessLevelOuter gl_TessLevelInner gl_TessCoord
+    gl_InvocationID gl_PrimitiveIDIn gl_Layer gl_ViewportIndex gl_FrontFacing
+    gl_PointCoord gl_SampleID gl_SamplePosition gl_FragColor
+    gl_FragData gl_FragDepth gl_SampleMask
+    gl_NumWorkGroups gl_WorkGroupSize gl_WorkGroupID gl_LocalInvocationID gl_GlobalInvocationID gl_LocalInvocationIndex
+    local_size_x local_size_y local_size_z
+
+    coherent volatile restrict readonly writeonly
+    image1D image2D image3D image2DRect imageCube imageBuffer image1DArray image2DArray imageCubeArray image2DMS image2DMSArray
+    uimage1D uimage2D uimage3D uimage2DRect uimageCube uimageBuffer uimage1DArray uimage2DArray uimageCubeArray uimage2DMS uimage2DMSArray
+    iimage1D iimage2D iimage3D iimage2DRect iimageCube iimageBuffer iimage1DArray iimage2DArray iimageCubeArray iimage2DMS iimage2DMSArray
+    size1x8 size1x16 size1x32 size2x32 size4x32 rgba32f rgba16f rg32f rg16f r32f r16f rgba8 rgba16 r11f_g11f_b10f rgb10_a2ui
+    rgb10_a2i rg16 rg8 r16 r8 rgba32i rgba16i rgba8i rg32i rg16i rg8i r32i r16i r8i rgba32ui rgba16ui rgba8ui rg32ui rg16ui rg8ui
+    r32ui r16ui r8ui rgba16_snorm rgba8_snorm rg16_snorm rg8_snorm r16_snorm r8_snorm
 ]]
 
 -- keywords - shouldn't be left out
 for w in keyw:gmatch("([a-zA-Z_0-9]+)") do
-	api[w] = {type="keyword"}
+    api[w] = {type="keyword"}
 end
 
 return api

api/lua/anttweakbar.lua

@@ -0,0 +1,553 @@
+--[[// tw tweakbar | AntTweakBar UI
+      enum { TW_VERSION = 114 }
+
+      typedef enum ETwType {
+	 TW_TYPE_UNDEF,
+	 TW_TYPE_BOOLCPP,
+	 TW_TYPE_BOOL8,
+	 TW_TYPE_BOOL16,
+	 TW_TYPE_BOOL32,
+	 TW_TYPE_CHAR,
+	 TW_TYPE_INT8,
+	 TW_TYPE_UINT8,
+	 TW_TYPE_INT16,
+	 TW_TYPE_UINT16,
+	 TW_TYPE_INT32,
+	 TW_TYPE_UINT32,
+	 TW_TYPE_FLOAT,
+	 TW_TYPE_DOUBLE,
+	 TW_TYPE_COLOR32,    // 32 bits color. Order is RGBA if API is OpenGL or Direct3D10, and inversed if API is Direct3D9 (can be modified by defining 'colorOrder=...', see doc)
+	 TW_TYPE_COLOR3F,    // 3 floats color. Order is RGB.
+	 TW_TYPE_COLOR4F,    // 4 floats color. Order is RGBA.
+	 TW_TYPE_CDSTRING,   // Null-terminated C Dynamic String (pointer to an array of char dynamically allocated with malloc/realloc/strdup)
+	 TW_TYPE__TEMP1,     // 
+	 TW_TYPE_QUAT4F,     // 4 floats encoding a quaternion {qx,qy,qz,qs}
+	 TW_TYPE_QUAT4D,     // 4 doubles encoding a quaternion {qx,qy,qz,qs}
+	 TW_TYPE_DIR3F,      // direction vector represented by 3 floats
+	 TW_TYPE_DIR3D,      // direction vector represented by 3 doubles
+	 TW_TYPE_CSSTRING_LEN0   = 0x30000000,
+	 TW_TYPE_CSSTRING_LEN256 = 0x30000000 + 256,
+      } TwType;
+
+      typedef struct CTwEnumVal  {
+	 int           Value;
+	 const char *  Label;
+      } TwEnumVal;
+
+      typedef struct CTwStructMember  {
+	 const char *  Name;
+	 TwType        Type;
+	 size_t        Offset;
+	 const char *  DefString;
+      } TwStructMember;
+      
+      typedef enum ETwParamValueType {
+	 TW_PARAM_INT32,
+	 TW_PARAM_FLOAT,
+	 TW_PARAM_DOUBLE,
+	 TW_PARAM_CSTRING // Null-terminated array of char (ie, c-string)
+      } TwParamValueType;
+      
+      typedef enum ETwGraphAPI {
+	 TW_OPENGL     = 1,
+	 TW_DIRECT3D9  = 2,
+	 TW_DIRECT3D10 = 3,
+	 TW_DIRECT3D11 = 4
+      } TwGraphAPI;
+      
+      typedef enum ETwKeyModifier {
+	 TW_KMOD_NONE  = 0x0000,   // same codes as SDL keysym.mod
+	 TW_KMOD_SHIFT = 0x0003,
+	 TW_KMOD_CTRL  = 0x00c0,
+	 TW_KMOD_ALT   = 0x0100,
+	 TW_KMOD_META  = 0x0c00
+      } TwKeyModifier;
+       
+      typedef enum EKeySpecial {
+	 TW_KEY_BACKSPACE  = '\b',
+	 TW_KEY_TAB        = '\t',
+	 TW_KEY_CLEAR      = 0x0c,
+	 TW_KEY_RETURN     = '\r',
+	 TW_KEY_PAUSE      = 0x13,
+	 TW_KEY_ESCAPE     = 0x1b,
+	 TW_KEY_SPACE      = ' ',
+	 TW_KEY_DELETE     = 0x7f,
+	 TW_KEY_UP         = 273,
+	 TW_KEY_DOWN,
+	 TW_KEY_RIGHT,
+	 TW_KEY_LEFT,
+	 TW_KEY_INSERT,
+	 TW_KEY_HOME,
+	 TW_KEY_END,
+	 TW_KEY_PAGE_UP,
+	 TW_KEY_PAGE_DOWN,
+	 TW_KEY_F1,
+	 TW_KEY_F2,
+	 TW_KEY_F3,
+	 TW_KEY_F4,
+	 TW_KEY_F5,
+	 TW_KEY_F6,
+	 TW_KEY_F7,
+	 TW_KEY_F8,
+	 TW_KEY_F9,
+	 TW_KEY_F10,
+	 TW_KEY_F11,
+	 TW_KEY_F12,
+	 TW_KEY_F13,
+	 TW_KEY_F14,
+	 TW_KEY_F15,
+	 TW_KEY_LAST
+      } TwKeySpecial;
+
+      typedef enum ETwMouseAction {
+	 TW_MOUSE_RELEASED,
+	 TW_MOUSE_PRESSED  
+      } TwMouseAction;
+      
+      typedef enum ETwMouseButtonID {
+	 TW_MOUSE_LEFT   = 1,
+	 TW_MOUSE_MIDDLE = 2,
+	 TW_MOUSE_RIGHT  = 3 
+      } TwMouseButtonID;
+
+      typedef void (*TwSetVarCallback)       ( const void *value, void *clientData );
+      typedef void (*TwGetVarCallback)       (       void *value, void *clientData );
+      typedef void (*TwButtonCallback)       (                    void *clientData );
+      typedef void (*TwSummaryCallback)      ( char *summaryString, size_t summaryMaxLength, const void *value, void *clientData );
+      typedef void (*TwCopyCDStringToClient) ( char **destinationClientStringPtr, const char *sourceString );
+      typedef void (*TwErrorHandler)         ( const char *errorMessage );
+      typedef void (*TwGLUTmousebuttonfun)   ( int glutButton, int glutState, int mouseX, int mouseY );
+      typedef void (*TwGLUTmousemotionfun)   ( int mouseX, int mouseY );
+      typedef void (*TwGLUTkeyboardfun)      ( unsigned char glutKey, int mouseX, int mouseY );
+      typedef void (*TwGLUTspecialfun)       ( int glutKey, int mouseX, int mouseY );
+      
+      typedef struct CTwBar TwBar;
+
+      TwBar*      TwNewBar(                    const char *barName );
+      int         TwDeleteBar(                 TwBar *bar          );
+      int         TwDeleteAllBars(                                 );
+      int         TwSetTopBar(                 const TwBar *bar    );
+      TwBar*      TwGetTopBar(                                     );
+      int         TwSetBottomBar(              const TwBar *bar    );
+      TwBar*      TwGetBottomBar(                                  );
+      const char* TwGetBarName(                TwBar *bar          );
+      int         TwGetBarCount(                                   );
+      TwBar*      TwGetBarByIndex(             int barIndex        );
+      TwBar*      TwGetBarByName(              const char *barName );
+      int         TwRefreshBar(                TwBar *bar          );
+      int         TwAddVarRW(                  TwBar *bar, const char *name, TwType type, void *var, const char *def );
+      int         TwAddVarRO(                  TwBar *bar, const char *name, TwType type, const void *var, const char *def );
+      int         TwAddVarCB(                  TwBar *bar, const char *name, TwType type, TwSetVarCallback setCallback, TwGetVarCallback getCallback, void *clientData, const char *def );
+      int         TwAddButton(                 TwBar *bar, const char *name, TwButtonCallback callback, void *clientData, const char *def );
+      int         TwAddSeparator(              TwBar *bar, const char *name, const char *def );
+      int         TwRemoveVar(                 TwBar *bar, const char *name );
+      int         TwRemoveAllVars(             TwBar *bar );
+      int         TwDefine(                    const char *def );
+      TwType      TwDefineEnum(                const char *name, const TwEnumVal *enumValues, unsigned int nbValues );
+      TwType      TwDefineEnumFromString(      const char *name, const char *enumString );
+      TwType      TwDefineStruct(              const char *name, const TwStructMember *structMembers, unsigned int nbMembers, size_t structSize, TwSummaryCallback summaryCallback, void *summaryClientData );
+      void        TwCopyCDStringToClientFunc(  TwCopyCDStringToClient copyCDStringFunc );
+      void        TwCopyCDStringToLibrary(     char **destinationLibraryStringPtr, const char *sourceClientString );
+      int         TwGetParam(                  TwBar *bar, const char *varName, const char *paramName, TwParamValueType paramValueType, unsigned int outValueMaxCount,      void *outValues );
+      int         TwSetParam(                  TwBar *bar, const char *varName, const char *paramName, TwParamValueType paramValueType, unsigned int     inValueCount, const void *inValues );
+      int         TwInit(                      TwGraphAPI graphAPI, void *device );
+      int         TwTerminate();
+      int         TwDraw();
+      int         TwWindowSize(                int width, int height);
+      int         TwSetCurrentWindow(          int windowID); // multi-windows support
+      int         TwGetCurrentWindow();
+      int         TwWindowExists(              int windowID);
+      int         TwKeyPressed(                int key, int modifiers);
+      int         TwKeyTest(                   int key, int modifiers);
+      int         TwMouseButton(               TwMouseAction action, TwMouseButtonID button);
+      int         TwMouseMotion(               int mouseX, int mouseY);
+      int         TwMouseWheel(                int pos);
+      const char* TwGetLastError(              );
+      void        TwHandleErrors(              TwErrorHandler errorHandler);
+      int         TwEventSDL(                  const void *sdlEvent, unsigned char sdlMajorVersion, unsigned char sdlMinorVersion);
+      int         TwEventSDL12(                const void *sdlEvent);
+      int         TwEventSDL13(                const void *sdlEvent);
+      int         TwEventMouseButtonGLFW(      int glfwButton, int glfwAction );
+      int         TwEventKeyGLFW(              int glfwKey,    int glfwAction );
+      int         TwEventCharGLFW(             int glfwChar,   int glfwAction );
+      int         TwEventMouseButtonGLUT(      int glutButton, int glutState, int mouseX, int mouseY);
+      int         TwEventMouseMotionGLUT(      int mouseX,     int mouseY);
+      int         TwEventKeyboardGLUT(         unsigned char glutKey, int mouseX, int mouseY );
+      int         TwEventSpecialGLUT(          int           glutKey, int mouseX, int mouseY );
+      int         TwGLUTModifiersFunc(         int (*glutGetModifiersFunc)(void) );
+      int         TwEventSFML(                 const void *sfmlEvent, unsigned char sfmlMajorVersion, unsigned char sfmlMinorVersion);
+]]
+  --auto-generated api from ffi headers
+  local api =
+         {
+         ["TW_TYPE_UNDEF"] = { type ='value', },
+       ["TW_TYPE_BOOLCPP"] = { type ='value', },
+       ["TW_TYPE_BOOL8"] = { type ='value', },
+       ["TW_TYPE_BOOL16"] = { type ='value', },
+       ["TW_TYPE_BOOL32"] = { type ='value', },
+       ["TW_TYPE_CHAR"] = { type ='value', },
+       ["TW_TYPE_INT8"] = { type ='value', },
+       ["TW_TYPE_UINT8"] = { type ='value', },
+       ["TW_TYPE_INT16"] = { type ='value', },
+       ["TW_TYPE_UINT16"] = { type ='value', },
+       ["TW_TYPE_INT32"] = { type ='value', },
+       ["TW_TYPE_UINT32"] = { type ='value', },
+       ["TW_TYPE_FLOAT"] = { type ='value', },
+       ["TW_TYPE_DOUBLE"] = { type ='value', },
+       ["TW_TYPE_COLOR32"] = { type ='value', },
+       ["TW_TYPE_COLOR3F"] = { type ='value', },
+       ["TW_TYPE_COLOR4F"] = { type ='value', },
+       ["TW_TYPE_CDSTRING"] = { type ='value', },
+       ["TW_TYPE__TEMP1"] = { type ='value', },
+       ["TW_TYPE_QUAT4F"] = { type ='value', },
+       ["TW_TYPE_QUAT4D"] = { type ='value', },
+       ["TW_TYPE_DIR3F"] = { type ='value', },
+       ["TW_TYPE_DIR3D"] = { type ='value', },
+       ["TW_TYPE_CSSTRING_LEN0"] = { type ='value', },
+       ["TW_TYPE_CSSTRING_LEN256"] = { type ='value', },
+       ["TW_PARAM_INT32"] = { type ='value', },
+       ["TW_PARAM_FLOAT"] = { type ='value', },
+       ["TW_PARAM_DOUBLE"] = { type ='value', },
+       ["TW_PARAM_CSTRING"] = { type ='value', },
+       ["TW_OPENGL"] = { type ='value', },
+       ["TW_DIRECT3D9"] = { type ='value', },
+       ["TW_DIRECT3D10"] = { type ='value', },
+       ["TW_DIRECT3D11"] = { type ='value', },
+       ["TW_KMOD_NONE"] = { type ='value', },
+       ["TW_KMOD_SHIFT"] = { type ='value', },
+       ["TW_KMOD_CTRL"] = { type ='value', },
+       ["TW_KMOD_ALT"] = { type ='value', },
+       ["TW_KMOD_META"] = { type ='value', },
+       ["TW_KEY_BACKSPACE"] = { type ='value', },
+       ["TW_KEY_TAB"] = { type ='value', },
+       ["TW_KEY_CLEAR"] = { type ='value', },
+       ["TW_KEY_RETURN"] = { type ='value', },
+       ["TW_KEY_PAUSE"] = { type ='value', },
+       ["TW_KEY_ESCAPE"] = { type ='value', },
+       ["TW_KEY_SPACE"] = { type ='value', },
+       ["TW_KEY_DELETE"] = { type ='value', },
+       ["TW_KEY_UP"] = { type ='value', },
+       ["TW_KEY_DOWN"] = { type ='value', },
+       ["TW_KEY_RIGHT"] = { type ='value', },
+       ["TW_KEY_LEFT"] = { type ='value', },
+       ["TW_KEY_INSERT"] = { type ='value', },
+       ["TW_KEY_HOME"] = { type ='value', },
+       ["TW_KEY_END"] = { type ='value', },
+       ["TW_KEY_PAGE_UP"] = { type ='value', },
+       ["TW_KEY_PAGE_DOWN"] = { type ='value', },
+       ["TW_KEY_F1"] = { type ='value', },
+       ["TW_KEY_F2"] = { type ='value', },
+       ["TW_KEY_F3"] = { type ='value', },
+       ["TW_KEY_F4"] = { type ='value', },
+       ["TW_KEY_F5"] = { type ='value', },
+       ["TW_KEY_F6"] = { type ='value', },
+       ["TW_KEY_F7"] = { type ='value', },
+       ["TW_KEY_F8"] = { type ='value', },
+       ["TW_KEY_F9"] = { type ='value', },
+       ["TW_KEY_F10"] = { type ='value', },
+       ["TW_KEY_F11"] = { type ='value', },
+       ["TW_KEY_F12"] = { type ='value', },
+       ["TW_KEY_F13"] = { type ='value', },
+       ["TW_KEY_F14"] = { type ='value', },
+       ["TW_KEY_F15"] = { type ='value', },
+       ["TW_KEY_LAST"] = { type ='value', },
+       ["TW_MOUSE_RELEASED"] = { type ='value', },
+       ["TW_MOUSE_PRESSED"] = { type ='value', },
+       ["TW_MOUSE_LEFT"] = { type ='value', },
+       ["TW_MOUSE_MIDDLE"] = { type ='value', },
+       ["TW_MOUSE_RIGHT"] = { type ='value', },
+       ["TwNewBar"] = { type ='function',
+          description = "",
+          returns = "(TwBar*)",
+          valuetype = nil,
+          args = "(const char *barName)", },
+       ["TwDeleteBar"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(TwBar *bar)", },
+       ["TwDeleteAllBars"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "()", },
+       ["TwSetTopBar"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(const TwBar *bar)", },
+       ["TwGetTopBar"] = { type ='function',
+          description = "",
+          returns = "(TwBar*)",
+          valuetype = nil,
+          args = "()", },
+       ["TwSetBottomBar"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(const TwBar *bar)", },
+       ["TwGetBottomBar"] = { type ='function',
+          description = "",
+          returns = "(TwBar*)",
+          valuetype = nil,
+          args = "()", },
+       ["TwGetBarName"] = { type ='function',
+          description = "",
+          returns = "(const char*)",
+          valuetype = "string",
+          args = "(TwBar *bar)", },
+       ["TwGetBarCount"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "()", },
+       ["TwGetBarByIndex"] = { type ='function',
+          description = "",
+          returns = "(TwBar*)",
+          valuetype = nil,
+          args = "(int barIndex)", },
+       ["TwGetBarByName"] = { type ='function',
+          description = "",
+          returns = "(TwBar*)",
+          valuetype = nil,
+          args = "(const char *barName)", },
+       ["TwRefreshBar"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(TwBar *bar)", },
+       ["TwAddVarRW"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(TwBar *bar, const char *name, TwType type, void *var, const char *def)", },
+       ["TwAddVarRO"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(TwBar *bar, const char *name, TwType type, const void *var, const char *def)", },
+       ["TwAddVarCB"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(TwBar *bar, const char *name, TwType type, TwSetVarCallback setCallback, TwGetVarCallback getCallback, void *clientData, const char *def)", },
+       ["TwAddButton"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(TwBar *bar, const char *name, TwButtonCallback callback, void *clientData, const char *def)", },
+       ["TwAddSeparator"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(TwBar *bar, const char *name, const char *def)", },
+       ["TwRemoveVar"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(TwBar *bar, const char *name)", },
+       ["TwRemoveAllVars"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(TwBar *bar)", },
+       ["TwDefine"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(const char *def)", },
+       ["TwDefineEnum"] = { type ='function',
+          description = "",
+          returns = "(TwType)",
+          valuetype = nil,
+          args = "(const char *name, const TwEnumVal *enumValues, unsigned int nbValues)", },
+       ["TwDefineEnumFromString"] = { type ='function',
+          description = "",
+          returns = "(TwType)",
+          valuetype = nil,
+          args = "(const char *name, const char *enumString)", },
+       ["TwDefineStruct"] = { type ='function',
+          description = "",
+          returns = "(TwType)",
+          valuetype = nil,
+          args = "(const char *name, const TwStructMember *structMembers, unsigned int nbMembers, size_t structSize, TwSummaryCallback summaryCallback, void *summaryClientData)", },
+       ["TwCopyCDStringToClientFunc"] = { type ='function',
+          description = "",
+          returns = "(void)",
+          valuetype = nil,
+          args = "(TwCopyCDStringToClient copyCDStringFunc)", },
+       ["TwCopyCDStringToLibrary"] = { type ='function',
+          description = "",
+          returns = "(void)",
+          valuetype = nil,
+          args = "(char **destinationLibraryStringPtr, const char *sourceClientString)", },
+       ["TwGetParam"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(TwBar *bar, const char *varName, const char *paramName, TwParamValueType paramValueType, unsigned int outValueMaxCount,      void *outValues)", },
+       ["TwSetParam"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(TwBar *bar, const char *varName, const char *paramName, TwParamValueType paramValueType, unsigned int     inValueCount, const void *inValues)", },
+       ["TwInit"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(TwGraphAPI graphAPI, void *device)", },
+       ["TwTerminate"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "()", },
+       ["TwDraw"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "()", },
+       ["TwWindowSize"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(int width, int height)", },
+       ["TwSetCurrentWindow"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(int windowID)", },
+       ["TwGetCurrentWindow"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "()", },
+       ["TwWindowExists"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(int windowID)", },
+       ["TwKeyPressed"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(int key, int modifiers)", },
+       ["TwKeyTest"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(int key, int modifiers)", },
+       ["TwMouseButton"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(TwMouseAction action, TwMouseButtonID button)", },
+       ["TwMouseMotion"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(int mouseX, int mouseY)", },
+       ["TwMouseWheel"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(int pos)", },
+       ["TwGetLastError"] = { type ='function',
+          description = "",
+          returns = "(const char*)",
+          valuetype = "string",
+          args = "()", },
+       ["TwHandleErrors"] = { type ='function',
+          description = "",
+          returns = "(void)",
+          valuetype = nil,
+          args = "(TwErrorHandler errorHandler)", },
+       ["TwEventSDL"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(const void *sdlEvent, unsigned char sdlMajorVersion, unsigned char sdlMinorVersion)", },
+       ["TwEventSDL12"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(const void *sdlEvent)", },
+       ["TwEventSDL13"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(const void *sdlEvent)", },
+       ["TwEventMouseButtonGLFW"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(int glfwButton, int glfwAction)", },
+       ["TwEventKeyGLFW"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(int glfwKey,    int glfwAction)", },
+       ["TwEventCharGLFW"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(int glfwChar,   int glfwAction)", },
+       ["TwEventMouseButtonGLUT"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(int glutButton, int glutState, int mouseX, int mouseY)", },
+       ["TwEventMouseMotionGLUT"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(int mouseX,     int mouseY)", },
+       ["TwEventKeyboardGLUT"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(unsigned char glutKey, int mouseX, int mouseY)", },
+       ["TwEventSpecialGLUT"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(int           glutKey, int mouseX, int mouseY)", },
+       ["glutGetModifiersFunc"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(void))", },
+       ["TwEventSFML"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(const void *sfmlEvent, unsigned char sfmlMajorVersion, unsigned char sfmlMinorVersion)", },
+       ["TwEnumVal"] = { type ='class',
+          description = "",
+          childs =         {
+          ["Value"] = { type ='value', description = "int          ", valuetype = nil, },
+        ["Label"] = { type ='value', description = "const char * ", valuetype = "string", },
+                }
+         },
+       ["TwStructMember"] = { type ='class',
+          description = "",
+          childs =         {
+          ["Name"] = { type ='value', description = "const char * ", valuetype = "string", },
+        ["Type"] = { type ='value', description = "TwType       ", valuetype = nil, },
+        ["Offset"] = { type ='value', description = "size_t       ", valuetype = nil, },
+        ["DefString"] = { type ='value', description = "const char * ", valuetype = "string", },
+                }
+         },
+               }
+  return {
+        tw = {
+      type = 'lib',
+      description = "AntTweakBar UI",
+      childs = api,
+    },
+        tweakbar = {
+      type = 'lib',
+      description = "AntTweakBar UI",
+      childs = api,
+    },
+      }
+  

api/lua/glfw.lua

@@ -0,0 +1,930 @@
+--[[// glfw | GLFW window manager
+enum {
+/*************************************************************************
+ * GLFW version
+ *************************************************************************/
+
+  GLFW_VERSION_MAJOR    =2,
+  GLFW_VERSION_MINOR    =7,
+  GLFW_VERSION_REVISION =2,
+
+
+/*************************************************************************
+ * Input handling definitions
+ *************************************************************************/
+
+/* Key and button state/action definitions */
+  GLFW_RELEASE            =0,
+  GLFW_PRESS              =1,
+  
+  GLFW_TRUE  = 1,
+  GLFW_FALSE = 0,
+
+/* Keyboard key definitions: 8-bit ISO-8859-1 (Latin 1) encoding is used
+ * for printable keys (such as A-Z, 0-9 etc), and values above 256
+ * represent special (non-printable) keys (e.g. F1, Page Up etc).
+ */
+  GLFW_KEY_UNKNOWN      =-1,
+  GLFW_KEY_SPACE        =32,
+  GLFW_KEY_APOSTROPHE        = 39, 
+  GLFW_KEY_COMMA             = 44, 
+  GLFW_KEY_MINUS             = 45, 
+  GLFW_KEY_PERIOD            = 46, 
+  GLFW_KEY_SLASH             = 47, 
+  GLFW_KEY_0                 = 48,
+  GLFW_KEY_1                 = 49,
+  GLFW_KEY_2                 = 50,
+  GLFW_KEY_3                 = 51,
+  GLFW_KEY_4                 = 52,
+  GLFW_KEY_5                 = 53,
+  GLFW_KEY_6                 = 54,
+  GLFW_KEY_7                 = 55,
+  GLFW_KEY_8                 = 56,
+  GLFW_KEY_9                 = 57,
+  GLFW_KEY_SEMICOLON         = 59, 
+  GLFW_KEY_EQUAL             = 61, 
+  GLFW_KEY_A                 = 65,
+  GLFW_KEY_B                 = 66,
+  GLFW_KEY_C                 = 67,
+  GLFW_KEY_D                 = 68,
+  GLFW_KEY_E                 = 69,
+  GLFW_KEY_F                 = 70,
+  GLFW_KEY_G                 = 71,
+  GLFW_KEY_H                 = 72,
+  GLFW_KEY_I                 = 73,
+  GLFW_KEY_J                 = 74,
+  GLFW_KEY_K                 = 75,
+  GLFW_KEY_L                 = 76,
+  GLFW_KEY_M                 = 77,
+  GLFW_KEY_N                 = 78,
+  GLFW_KEY_O                 = 79,
+  GLFW_KEY_P                 = 80,
+  GLFW_KEY_Q                 = 81,
+  GLFW_KEY_R                 = 82,
+  GLFW_KEY_S                 = 83,
+  GLFW_KEY_T                 = 84,
+  GLFW_KEY_U                 = 85,
+  GLFW_KEY_V                 = 86,
+  GLFW_KEY_W                 = 87,
+  GLFW_KEY_X                 = 88,
+  GLFW_KEY_Y                 = 89,
+  GLFW_KEY_Z                 = 90,
+  GLFW_KEY_LEFT_BRACKET      = 91, 
+  GLFW_KEY_BACKSLASH         = 92,
+  GLFW_KEY_RIGHT_BRACKET     = 93, 
+  GLFW_KEY_GRAVE_ACCENT      = 96,
+  GLFW_KEY_WORLD_1           = 161,
+  GLFW_KEY_WORLD_2           = 162,
+
+  GLFW_KEY_SPECIAL      =256,
+  GLFW_KEY_ESC          =(GLFW_KEY_SPECIAL+1),
+  GLFW_KEY_F1           =(GLFW_KEY_SPECIAL+2),
+  GLFW_KEY_F2           =(GLFW_KEY_SPECIAL+3),
+  GLFW_KEY_F3           =(GLFW_KEY_SPECIAL+4),
+  GLFW_KEY_F4           =(GLFW_KEY_SPECIAL+5),
+  GLFW_KEY_F5           =(GLFW_KEY_SPECIAL+6),
+  GLFW_KEY_F6           =(GLFW_KEY_SPECIAL+7),
+  GLFW_KEY_F7           =(GLFW_KEY_SPECIAL+8),
+  GLFW_KEY_F8           =(GLFW_KEY_SPECIAL+9),
+  GLFW_KEY_F9           =(GLFW_KEY_SPECIAL+10),
+  GLFW_KEY_F10          =(GLFW_KEY_SPECIAL+11),
+  GLFW_KEY_F11          =(GLFW_KEY_SPECIAL+12),
+  GLFW_KEY_F12          =(GLFW_KEY_SPECIAL+13),
+  GLFW_KEY_F13          =(GLFW_KEY_SPECIAL+14),
+  GLFW_KEY_F14          =(GLFW_KEY_SPECIAL+15),
+  GLFW_KEY_F15          =(GLFW_KEY_SPECIAL+16),
+  GLFW_KEY_F16          =(GLFW_KEY_SPECIAL+17),
+  GLFW_KEY_F17          =(GLFW_KEY_SPECIAL+18),
+  GLFW_KEY_F18          =(GLFW_KEY_SPECIAL+19),
+  GLFW_KEY_F19          =(GLFW_KEY_SPECIAL+20),
+  GLFW_KEY_F20          =(GLFW_KEY_SPECIAL+21),
+  GLFW_KEY_F21          =(GLFW_KEY_SPECIAL+22),
+  GLFW_KEY_F22          =(GLFW_KEY_SPECIAL+23),
+  GLFW_KEY_F23          =(GLFW_KEY_SPECIAL+24),
+  GLFW_KEY_F24          =(GLFW_KEY_SPECIAL+25),
+  GLFW_KEY_F25          =(GLFW_KEY_SPECIAL+26),
+  GLFW_KEY_UP           =(GLFW_KEY_SPECIAL+27),
+  GLFW_KEY_DOWN         =(GLFW_KEY_SPECIAL+28),
+  GLFW_KEY_LEFT         =(GLFW_KEY_SPECIAL+29),
+  GLFW_KEY_RIGHT        =(GLFW_KEY_SPECIAL+30),
+  GLFW_KEY_LSHIFT       =(GLFW_KEY_SPECIAL+31),
+  GLFW_KEY_RSHIFT       =(GLFW_KEY_SPECIAL+32),
+  GLFW_KEY_LCTRL        =(GLFW_KEY_SPECIAL+33),
+  GLFW_KEY_RCTRL        =(GLFW_KEY_SPECIAL+34),
+  GLFW_KEY_LALT         =(GLFW_KEY_SPECIAL+35),
+  GLFW_KEY_RALT         =(GLFW_KEY_SPECIAL+36),
+  GLFW_KEY_TAB          =(GLFW_KEY_SPECIAL+37),
+  GLFW_KEY_ENTER        =(GLFW_KEY_SPECIAL+38),
+  GLFW_KEY_BACKSPACE    =(GLFW_KEY_SPECIAL+39),
+  GLFW_KEY_INSERT       =(GLFW_KEY_SPECIAL+40),
+  GLFW_KEY_DEL          =(GLFW_KEY_SPECIAL+41),
+  GLFW_KEY_PAGEUP       =(GLFW_KEY_SPECIAL+42),
+  GLFW_KEY_PAGEDOWN     =(GLFW_KEY_SPECIAL+43),
+  GLFW_KEY_HOME         =(GLFW_KEY_SPECIAL+44),
+  GLFW_KEY_END          =(GLFW_KEY_SPECIAL+45),
+  GLFW_KEY_KP_0         =(GLFW_KEY_SPECIAL+46),
+  GLFW_KEY_KP_1         =(GLFW_KEY_SPECIAL+47),
+  GLFW_KEY_KP_2         =(GLFW_KEY_SPECIAL+48),
+  GLFW_KEY_KP_3         =(GLFW_KEY_SPECIAL+49),
+  GLFW_KEY_KP_4         =(GLFW_KEY_SPECIAL+50),
+  GLFW_KEY_KP_5         =(GLFW_KEY_SPECIAL+51),
+  GLFW_KEY_KP_6         =(GLFW_KEY_SPECIAL+52),
+  GLFW_KEY_KP_7         =(GLFW_KEY_SPECIAL+53),
+  GLFW_KEY_KP_8         =(GLFW_KEY_SPECIAL+54),
+  GLFW_KEY_KP_9         =(GLFW_KEY_SPECIAL+55),
+  GLFW_KEY_KP_DIVIDE    =(GLFW_KEY_SPECIAL+56),
+  GLFW_KEY_KP_MULTIPLY  =(GLFW_KEY_SPECIAL+57),
+  GLFW_KEY_KP_SUBTRACT  =(GLFW_KEY_SPECIAL+58),
+  GLFW_KEY_KP_ADD       =(GLFW_KEY_SPECIAL+59),
+  GLFW_KEY_KP_DECIMAL   =(GLFW_KEY_SPECIAL+60),
+  GLFW_KEY_KP_EQUAL     =(GLFW_KEY_SPECIAL+61),
+  GLFW_KEY_KP_ENTER     =(GLFW_KEY_SPECIAL+62),
+  GLFW_KEY_KP_NUM_LOCK  =(GLFW_KEY_SPECIAL+63),
+  GLFW_KEY_CAPS_LOCK    =(GLFW_KEY_SPECIAL+64),
+  GLFW_KEY_SCROLL_LOCK  =(GLFW_KEY_SPECIAL+65),
+  GLFW_KEY_PAUSE        =(GLFW_KEY_SPECIAL+66),
+  GLFW_KEY_LSUPER       =(GLFW_KEY_SPECIAL+67),
+  GLFW_KEY_RSUPER       =(GLFW_KEY_SPECIAL+68),
+  GLFW_KEY_MENU         =(GLFW_KEY_SPECIAL+69),
+  GLFW_KEY_LAST         =GLFW_KEY_MENU,
+
+/* Mouse button definitions */
+  GLFW_MOUSE_BUTTON_1      =0,
+  GLFW_MOUSE_BUTTON_2      =1,
+  GLFW_MOUSE_BUTTON_3      =2,
+  GLFW_MOUSE_BUTTON_4      =3,
+  GLFW_MOUSE_BUTTON_5      =4,
+  GLFW_MOUSE_BUTTON_6      =5,
+  GLFW_MOUSE_BUTTON_7      =6,
+  GLFW_MOUSE_BUTTON_8      =7,
+  GLFW_MOUSE_BUTTON_LAST   =GLFW_MOUSE_BUTTON_8,
+
+/* Mouse button aliases */
+  GLFW_MOUSE_BUTTON_LEFT   =GLFW_MOUSE_BUTTON_1,
+  GLFW_MOUSE_BUTTON_RIGHT  =GLFW_MOUSE_BUTTON_2,
+  GLFW_MOUSE_BUTTON_MIDDLE =GLFW_MOUSE_BUTTON_3,
+
+
+/* Joystick identifiers */
+  GLFW_JOYSTICK_1          =0,
+  GLFW_JOYSTICK_2          =1,
+  GLFW_JOYSTICK_3          =2,
+  GLFW_JOYSTICK_4          =3,
+  GLFW_JOYSTICK_5          =4,
+  GLFW_JOYSTICK_6          =5,
+  GLFW_JOYSTICK_7          =6,
+  GLFW_JOYSTICK_8          =7,
+  GLFW_JOYSTICK_9          =8,
+  GLFW_JOYSTICK_10         =9,
+  GLFW_JOYSTICK_11         =10,
+  GLFW_JOYSTICK_12         =11,
+  GLFW_JOYSTICK_13         =12,
+  GLFW_JOYSTICK_14         =13,
+  GLFW_JOYSTICK_15         =14,
+  GLFW_JOYSTICK_16         =15,
+  GLFW_JOYSTICK_LAST       =GLFW_JOYSTICK_16,
+
+
+/*************************************************************************
+ * Other definitions
+ *************************************************************************/
+
+/* glfwOpenWindow modes */
+  GLFW_WINDOW               =0x00010001,
+  GLFW_FULLSCREEN           =0x00010002,
+
+/* glfwGetWindowParam tokens */
+  GLFW_OPENED               =0x00020001,
+  GLFW_ACTIVE               =0x00020002,
+  GLFW_ICONIFIED            =0x00020003,
+  GLFW_ACCELERATED          =0x00020004,
+  GLFW_RED_BITS             =0x00020005,
+  GLFW_GREEN_BITS           =0x00020006,
+  GLFW_BLUE_BITS            =0x00020007,
+  GLFW_ALPHA_BITS           =0x00020008,
+  GLFW_DEPTH_BITS           =0x00020009,
+  GLFW_STENCIL_BITS         =0x0002000A,
+
+/* The following constants are used for both glfwGetWindowParam
+ * and glfwOpenWindowHint
+ */
+  GLFW_REFRESH_RATE         =0x0002000B,
+  GLFW_ACCUM_RED_BITS       =0x0002000C,
+  GLFW_ACCUM_GREEN_BITS     =0x0002000D,
+  GLFW_ACCUM_BLUE_BITS      =0x0002000E,
+  GLFW_ACCUM_ALPHA_BITS     =0x0002000F,
+  GLFW_AUX_BUFFERS          =0x00020010,
+  GLFW_STEREO               =0x00020011,
+  GLFW_WINDOW_NO_RESIZE     =0x00020012,
+  GLFW_FSAA_SAMPLES         =0x00020013,
+  GLFW_OPENGL_VERSION_MAJOR =0x00020014,
+  GLFW_OPENGL_VERSION_MINOR =0x00020015,
+  GLFW_OPENGL_FORWARD_COMPAT =0x00020016,
+  GLFW_OPENGL_DEBUG_CONTEXT =0x00020017,
+  GLFW_OPENGL_PROFILE       =0x00020018,
+
+/* GLFW_OPENGL_PROFILE tokens */
+  GLFW_OPENGL_CORE_PROFILE  =0x00050001,
+  GLFW_OPENGL_COMPAT_PROFILE =0x00050002,
+
+/* glfwEnable/glfwDisable tokens */
+  GLFW_MOUSE_CURSOR         =0x00030001,
+  GLFW_STICKY_KEYS          =0x00030002,
+  GLFW_STICKY_MOUSE_BUTTONS =0x00030003,
+  GLFW_SYSTEM_KEYS          =0x00030004,
+  GLFW_KEY_REPEAT           =0x00030005,
+  GLFW_AUTO_POLL_EVENTS     =0x00030006,
+
+/* glfwWaitThread wait modes */
+  GLFW_WAIT                 =0x00040001,
+  GLFW_NOWAIT               =0x00040002,
+
+/* glfwGetJoystickParam tokens */
+  GLFW_PRESENT              =0x00050001,
+  GLFW_AXES                 =0x00050002,
+  GLFW_BUTTONS              =0x00050003,
+
+/* glfwReadImage/glfwLoadTexture2D flags */
+  GLFW_NO_RESCALE_BIT       =0x00000001 /* Only for glfwReadImage */,
+  GLFW_ORIGIN_UL_BIT        =0x00000002,
+  GLFW_BUILD_MIPMAPS_BIT    =0x00000004 /* Only for glfwLoadTexture2D */,
+  GLFW_ALPHA_MAP_BIT        =0x00000008,
+
+/* Time spans longer than this (seconds) are considered to be infinity */
+};
+
+const float GLFW_INFINITY =100000.0;
+
+/* The video mode structure used by glfwGetVideoModes() */
+typedef struct {
+    int Width, Height;
+    int RedBits, BlueBits, GreenBits;
+} GLFWvidmode;
+
+/* Image/texture information */
+typedef struct {
+    int Width, Height;
+    int Format;
+    int BytesPerPixel;
+    unsigned char *Data;
+} GLFWimage;
+
+/* Thread ID */
+typedef int GLFWthread;
+
+/* Mutex object */
+typedef void * GLFWmutex;
+
+/* Condition variable object */
+typedef void * GLFWcond;
+
+/* Function pointer types */
+typedef void (GLFWCALL * GLFWwindowsizefun)(int,int);
+typedef int  (GLFWCALL * GLFWwindowclosefun)(void);
+typedef void (GLFWCALL * GLFWwindowrefreshfun)(void);
+typedef void (GLFWCALL * GLFWmousebuttonfun)(int,int);
+typedef void (GLFWCALL * GLFWmouseposfun)(int,int);
+typedef void (GLFWCALL * GLFWmousewheelfun)(int);
+typedef void (GLFWCALL * GLFWkeyfun)(int,int);
+typedef void (GLFWCALL * GLFWcharfun)(int,int);
+typedef void (GLFWCALL * GLFWthreadfun)(void *);
+
+
+/*************************************************************************
+ * Prototypes
+ *************************************************************************/
+
+/* GLFW initialization, termination and version querying */
+int  glfwInit( void );
+void glfwTerminate( void );
+void glfwGetVersion( int *major, int *minor, int *rev );
+
+/* Window handling */
+int  glfwOpenWindow( int width, int height, int redbits, int greenbits, int bluebits, int alphabits, int depthbits, int stencilbits, int mode );
+void glfwOpenWindowHint( int target, int hint );
+void glfwCloseWindow( void );
+void glfwSetWindowTitle( const char *title );
+void glfwGetWindowSize( int *width, int *height );
+void glfwSetWindowSize( int width, int height );
+void glfwSetWindowPos( int x, int y );
+void glfwIconifyWindow( void );
+void glfwRestoreWindow( void );
+void glfwSwapBuffers( void );
+void glfwSwapInterval( int interval );
+int  glfwGetWindowParam( int param );
+void glfwSetWindowSizeCallback( GLFWwindowsizefun cbfun );
+void glfwSetWindowCloseCallback( GLFWwindowclosefun cbfun );
+void glfwSetWindowRefreshCallback( GLFWwindowrefreshfun cbfun );
+
+/* Video mode functions */
+int  glfwGetVideoModes( GLFWvidmode *list, int maxcount );
+void glfwGetDesktopMode( GLFWvidmode *mode );
+
+/* Input handling */
+void glfwPollEvents( void );
+void glfwWaitEvents( void );
+int  glfwGetKey( int key );
+int  glfwGetMouseButton( int button );
+void glfwGetMousePos( int *xpos, int *ypos );
+void glfwSetMousePos( int xpos, int ypos );
+int  glfwGetMouseWheel( void );
+void glfwSetMouseWheel( int pos );
+void glfwSetKeyCallback( GLFWkeyfun cbfun );
+void glfwSetCharCallback( GLFWcharfun cbfun );
+void glfwSetMouseButtonCallback( GLFWmousebuttonfun cbfun );
+void glfwSetMousePosCallback( GLFWmouseposfun cbfun );
+void glfwSetMouseWheelCallback( GLFWmousewheelfun cbfun );
+
+/* Joystick input */
+int glfwGetJoystickParam( int joy, int param );
+int glfwGetJoystickPos( int joy, float *pos, int numaxes );
+int glfwGetJoystickButtons( int joy, unsigned char *buttons, int numbuttons );
+
+/* Time */
+double glfwGetTime( void );
+void   glfwSetTime( double time );
+void   glfwSleep( double time );
+
+/* Extension support */
+int   glfwExtensionSupported( const char *extension );
+void* glfwGetProcAddress( const char *procname );
+void  glfwGetGLVersion( int *major, int *minor, int *rev );
+
+/* Threading support */
+GLFWthread glfwCreateThread( GLFWthreadfun fun, void *arg );
+void glfwDestroyThread( GLFWthread ID );
+int  glfwWaitThread( GLFWthread ID, int waitmode );
+GLFWthread glfwGetThreadID( void );
+GLFWmutex glfwCreateMutex( void );
+void glfwDestroyMutex( GLFWmutex mutex );
+void glfwLockMutex( GLFWmutex mutex );
+void glfwUnlockMutex( GLFWmutex mutex );
+GLFWcond glfwCreateCond( void );
+void glfwDestroyCond( GLFWcond cond );
+void glfwWaitCond( GLFWcond cond, GLFWmutex mutex, double timeout );
+void glfwSignalCond( GLFWcond cond );
+void glfwBroadcastCond( GLFWcond cond );
+int  glfwGetNumberOfProcessors( void );
+
+/* Enable/disable functions */
+void glfwEnable( int token );
+void glfwDisable( int token );
+
+/* Image/texture I/O support */
+int  glfwReadImage( const char *name, GLFWimage *img, int flags );
+int  glfwReadMemoryImage( const void *data, long size, GLFWimage *img, int flags );
+void glfwFreeImage( GLFWimage *img );
+int  glfwLoadTexture2D( const char *name, int flags );
+int  glfwLoadMemoryTexture2D( const void *data, long size, int flags );
+int  glfwLoadTextureImage2D( GLFWimage *img, int flags );
+]]
+  --auto-generated api from ffi headers
+  local api =
+         {
+         ["GLFW_VERSION_MAJOR"] = { type ='value', },
+       ["GLFW_VERSION_MINOR"] = { type ='value', },
+       ["GLFW_VERSION_REVISION"] = { type ='value', },
+       ["GLFW_RELEASE"] = { type ='value', },
+       ["GLFW_PRESS"] = { type ='value', },
+       ["GLFW_KEY_UNKNOWN"] = { type ='value', },
+       ["GLFW_KEY_SPACE"] = { type ='value', },
+       ["GLFW_KEY_APOSTROPHE"] = { type ='value', },
+       ["GLFW_KEY_COMMA"] = { type ='value', },
+       ["GLFW_KEY_MINUS"] = { type ='value', },
+       ["GLFW_KEY_PERIOD"] = { type ='value', },
+       ["GLFW_KEY_SLASH"] = { type ='value', },
+       ["GLFW_KEY_0"] = { type ='value', },
+       ["GLFW_KEY_1"] = { type ='value', },
+       ["GLFW_KEY_2"] = { type ='value', },
+       ["GLFW_KEY_3"] = { type ='value', },
+       ["GLFW_KEY_4"] = { type ='value', },
+       ["GLFW_KEY_5"] = { type ='value', },
+       ["GLFW_KEY_6"] = { type ='value', },
+       ["GLFW_KEY_7"] = { type ='value', },
+       ["GLFW_KEY_8"] = { type ='value', },
+       ["GLFW_KEY_9"] = { type ='value', },
+       ["GLFW_KEY_SEMICOLON"] = { type ='value', },
+       ["GLFW_KEY_EQUAL"] = { type ='value', },
+       ["GLFW_KEY_A"] = { type ='value', },
+       ["GLFW_KEY_B"] = { type ='value', },
+       ["GLFW_KEY_C"] = { type ='value', },
+       ["GLFW_KEY_D"] = { type ='value', },
+       ["GLFW_KEY_E"] = { type ='value', },
+       ["GLFW_KEY_F"] = { type ='value', },
+       ["GLFW_KEY_G"] = { type ='value', },
+       ["GLFW_KEY_H"] = { type ='value', },
+       ["GLFW_KEY_I"] = { type ='value', },
+       ["GLFW_KEY_J"] = { type ='value', },
+       ["GLFW_KEY_K"] = { type ='value', },
+       ["GLFW_KEY_L"] = { type ='value', },
+       ["GLFW_KEY_M"] = { type ='value', },
+       ["GLFW_KEY_N"] = { type ='value', },
+       ["GLFW_KEY_O"] = { type ='value', },
+       ["GLFW_KEY_P"] = { type ='value', },
+       ["GLFW_KEY_Q"] = { type ='value', },
+       ["GLFW_KEY_R"] = { type ='value', },
+       ["GLFW_KEY_S"] = { type ='value', },
+       ["GLFW_KEY_T"] = { type ='value', },
+       ["GLFW_KEY_U"] = { type ='value', },
+       ["GLFW_KEY_V"] = { type ='value', },
+       ["GLFW_KEY_W"] = { type ='value', },
+       ["GLFW_KEY_X"] = { type ='value', },
+       ["GLFW_KEY_Y"] = { type ='value', },
+       ["GLFW_KEY_Z"] = { type ='value', },
+       ["GLFW_KEY_LEFT_BRACKET"] = { type ='value', },
+       ["GLFW_KEY_BACKSLASH"] = { type ='value', },
+       ["GLFW_KEY_RIGHT_BRACKET"] = { type ='value', },
+       ["GLFW_KEY_GRAVE_ACCENT"] = { type ='value', },
+       ["GLFW_KEY_WORLD_1"] = { type ='value', },
+       ["GLFW_KEY_WORLD_2"] = { type ='value', },
+       ["GLFW_KEY_SPECIAL"] = { type ='value', },
+       ["GLFW_KEY_ESC"] = { type ='value', },
+       ["GLFW_KEY_F1"] = { type ='value', },
+       ["GLFW_KEY_F2"] = { type ='value', },
+       ["GLFW_KEY_F3"] = { type ='value', },
+       ["GLFW_KEY_F4"] = { type ='value', },
+       ["GLFW_KEY_F5"] = { type ='value', },
+       ["GLFW_KEY_F6"] = { type ='value', },
+       ["GLFW_KEY_F7"] = { type ='value', },
+       ["GLFW_KEY_F8"] = { type ='value', },
+       ["GLFW_KEY_F9"] = { type ='value', },
+       ["GLFW_KEY_F10"] = { type ='value', },
+       ["GLFW_KEY_F11"] = { type ='value', },
+       ["GLFW_KEY_F12"] = { type ='value', },
+       ["GLFW_KEY_F13"] = { type ='value', },
+       ["GLFW_KEY_F14"] = { type ='value', },
+       ["GLFW_KEY_F15"] = { type ='value', },
+       ["GLFW_KEY_F16"] = { type ='value', },
+       ["GLFW_KEY_F17"] = { type ='value', },
+       ["GLFW_KEY_F18"] = { type ='value', },
+       ["GLFW_KEY_F19"] = { type ='value', },
+       ["GLFW_KEY_F20"] = { type ='value', },
+       ["GLFW_KEY_F21"] = { type ='value', },
+       ["GLFW_KEY_F22"] = { type ='value', },
+       ["GLFW_KEY_F23"] = { type ='value', },
+       ["GLFW_KEY_F24"] = { type ='value', },
+       ["GLFW_KEY_F25"] = { type ='value', },
+       ["GLFW_KEY_UP"] = { type ='value', },
+       ["GLFW_KEY_DOWN"] = { type ='value', },
+       ["GLFW_KEY_LEFT"] = { type ='value', },
+       ["GLFW_KEY_RIGHT"] = { type ='value', },
+       ["GLFW_KEY_LSHIFT"] = { type ='value', },
+       ["GLFW_KEY_RSHIFT"] = { type ='value', },
+       ["GLFW_KEY_LCTRL"] = { type ='value', },
+       ["GLFW_KEY_RCTRL"] = { type ='value', },
+       ["GLFW_KEY_LALT"] = { type ='value', },
+       ["GLFW_KEY_RALT"] = { type ='value', },
+       ["GLFW_KEY_TAB"] = { type ='value', },
+       ["GLFW_KEY_ENTER"] = { type ='value', },
+       ["GLFW_KEY_BACKSPACE"] = { type ='value', },
+       ["GLFW_KEY_INSERT"] = { type ='value', },
+       ["GLFW_KEY_DEL"] = { type ='value', },
+       ["GLFW_KEY_PAGEUP"] = { type ='value', },
+       ["GLFW_KEY_PAGEDOWN"] = { type ='value', },
+       ["GLFW_KEY_HOME"] = { type ='value', },
+       ["GLFW_KEY_END"] = { type ='value', },
+       ["GLFW_KEY_KP_0"] = { type ='value', },
+       ["GLFW_KEY_KP_1"] = { type ='value', },
+       ["GLFW_KEY_KP_2"] = { type ='value', },
+       ["GLFW_KEY_KP_3"] = { type ='value', },
+       ["GLFW_KEY_KP_4"] = { type ='value', },
+       ["GLFW_KEY_KP_5"] = { type ='value', },
+       ["GLFW_KEY_KP_6"] = { type ='value', },
+       ["GLFW_KEY_KP_7"] = { type ='value', },
+       ["GLFW_KEY_KP_8"] = { type ='value', },
+       ["GLFW_KEY_KP_9"] = { type ='value', },
+       ["GLFW_KEY_KP_DIVIDE"] = { type ='value', },
+       ["GLFW_KEY_KP_MULTIPLY"] = { type ='value', },
+       ["GLFW_KEY_KP_SUBTRACT"] = { type ='value', },
+       ["GLFW_KEY_KP_ADD"] = { type ='value', },
+       ["GLFW_KEY_KP_DECIMAL"] = { type ='value', },
+       ["GLFW_KEY_KP_EQUAL"] = { type ='value', },
+       ["GLFW_KEY_KP_ENTER"] = { type ='value', },
+       ["GLFW_KEY_KP_NUM_LOCK"] = { type ='value', },
+       ["GLFW_KEY_CAPS_LOCK"] = { type ='value', },
+       ["GLFW_KEY_SCROLL_LOCK"] = { type ='value', },
+       ["GLFW_KEY_PAUSE"] = { type ='value', },
+       ["GLFW_KEY_LSUPER"] = { type ='value', },
+       ["GLFW_KEY_RSUPER"] = { type ='value', },
+       ["GLFW_KEY_MENU"] = { type ='value', },
+       ["GLFW_KEY_LAST"] = { type ='value', },
+       ["GLFW_MOUSE_BUTTON_1"] = { type ='value', },
+       ["GLFW_MOUSE_BUTTON_2"] = { type ='value', },
+       ["GLFW_MOUSE_BUTTON_3"] = { type ='value', },
+       ["GLFW_MOUSE_BUTTON_4"] = { type ='value', },
+       ["GLFW_MOUSE_BUTTON_5"] = { type ='value', },
+       ["GLFW_MOUSE_BUTTON_6"] = { type ='value', },
+       ["GLFW_MOUSE_BUTTON_7"] = { type ='value', },
+       ["GLFW_MOUSE_BUTTON_8"] = { type ='value', },
+       ["GLFW_MOUSE_BUTTON_LAST"] = { type ='value', },
+       ["GLFW_MOUSE_BUTTON_LEFT"] = { type ='value', },
+       ["GLFW_MOUSE_BUTTON_RIGHT"] = { type ='value', },
+       ["GLFW_MOUSE_BUTTON_MIDDLE"] = { type ='value', },
+       ["GLFW_JOYSTICK_1"] = { type ='value', },
+       ["GLFW_JOYSTICK_2"] = { type ='value', },
+       ["GLFW_JOYSTICK_3"] = { type ='value', },
+       ["GLFW_JOYSTICK_4"] = { type ='value', },
+       ["GLFW_JOYSTICK_5"] = { type ='value', },
+       ["GLFW_JOYSTICK_6"] = { type ='value', },
+       ["GLFW_JOYSTICK_7"] = { type ='value', },
+       ["GLFW_JOYSTICK_8"] = { type ='value', },
+       ["GLFW_JOYSTICK_9"] = { type ='value', },
+       ["GLFW_JOYSTICK_10"] = { type ='value', },
+       ["GLFW_JOYSTICK_11"] = { type ='value', },
+       ["GLFW_JOYSTICK_12"] = { type ='value', },
+       ["GLFW_JOYSTICK_13"] = { type ='value', },
+       ["GLFW_JOYSTICK_14"] = { type ='value', },
+       ["GLFW_JOYSTICK_15"] = { type ='value', },
+       ["GLFW_JOYSTICK_16"] = { type ='value', },
+       ["GLFW_JOYSTICK_LAST"] = { type ='value', },
+       ["GLFW_WINDOW"] = { type ='value', },
+       ["GLFW_FULLSCREEN"] = { type ='value', },
+       ["GLFW_OPENED"] = { type ='value', },
+       ["GLFW_ACTIVE"] = { type ='value', },
+       ["GLFW_ICONIFIED"] = { type ='value', },
+       ["GLFW_ACCELERATED"] = { type ='value', },
+       ["GLFW_RED_BITS"] = { type ='value', },
+       ["GLFW_GREEN_BITS"] = { type ='value', },
+       ["GLFW_BLUE_BITS"] = { type ='value', },
+       ["GLFW_ALPHA_BITS"] = { type ='value', },
+       ["GLFW_DEPTH_BITS"] = { type ='value', },
+       ["GLFW_STENCIL_BITS"] = { type ='value', },
+       ["GLFW_REFRESH_RATE"] = { type ='value', },
+       ["GLFW_ACCUM_RED_BITS"] = { type ='value', },
+       ["GLFW_ACCUM_GREEN_BITS"] = { type ='value', },
+       ["GLFW_ACCUM_BLUE_BITS"] = { type ='value', },
+       ["GLFW_ACCUM_ALPHA_BITS"] = { type ='value', },
+       ["GLFW_AUX_BUFFERS"] = { type ='value', },
+       ["GLFW_STEREO"] = { type ='value', },
+       ["GLFW_WINDOW_NO_RESIZE"] = { type ='value', },
+       ["GLFW_FSAA_SAMPLES"] = { type ='value', },
+       ["GLFW_OPENGL_VERSION_MAJOR"] = { type ='value', },
+       ["GLFW_OPENGL_VERSION_MINOR"] = { type ='value', },
+       ["GLFW_OPENGL_FORWARD_COMPAT"] = { type ='value', },
+       ["GLFW_OPENGL_DEBUG_CONTEXT"] = { type ='value', },
+       ["GLFW_OPENGL_PROFILE"] = { type ='value', },
+       ["GLFW_OPENGL_CORE_PROFILE"] = { type ='value', },
+       ["GLFW_OPENGL_COMPAT_PROFILE"] = { type ='value', },
+       ["GLFW_MOUSE_CURSOR"] = { type ='value', },
+       ["GLFW_STICKY_KEYS"] = { type ='value', },
+       ["GLFW_STICKY_MOUSE_BUTTONS"] = { type ='value', },
+       ["GLFW_SYSTEM_KEYS"] = { type ='value', },
+       ["GLFW_KEY_REPEAT"] = { type ='value', },
+       ["GLFW_AUTO_POLL_EVENTS"] = { type ='value', },
+       ["GLFW_WAIT"] = { type ='value', },
+       ["GLFW_NOWAIT"] = { type ='value', },
+       ["GLFW_PRESENT"] = { type ='value', },
+       ["GLFW_AXES"] = { type ='value', },
+       ["GLFW_BUTTONS"] = { type ='value', },
+       ["GLFW_NO_RESCALE_BIT"] = { type ='value', },
+       ["GLFW_ORIGIN_UL_BIT"] = { type ='value', },
+       ["GLFW_BUILD_MIPMAPS_BIT"] = { type ='value', },
+       ["GLFW_ALPHA_MAP_BIT"] = { type ='value', },
+       ["glfwInit"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(void)", },
+       ["glfwTerminate"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(void)", },
+       ["glfwGetVersion"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(int *major, int *minor, int *rev)", },
+       ["glfwOpenWindow"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(int width, int height, int redbits, int greenbits, int bluebits, int alphabits, int depthbits, int stencilbits, int mode)", },
+       ["glfwOpenWindowHint"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(int target, int hint)", },
+       ["glfwCloseWindow"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(void)", },
+       ["glfwSetWindowTitle"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(const char *title)", },
+       ["glfwGetWindowSize"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(int *width, int *height)", },
+       ["glfwSetWindowSize"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(int width, int height)", },
+       ["glfwSetWindowPos"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(int x, int y)", },
+       ["glfwIconifyWindow"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(void)", },
+       ["glfwRestoreWindow"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(void)", },
+       ["glfwSwapBuffers"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(void)", },
+       ["glfwSwapInterval"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(int interval)", },
+       ["glfwGetWindowParam"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(int param)", },
+       ["glfwSetWindowSizeCallback"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(GLFWwindowsizefun cbfun)", },
+       ["glfwSetWindowCloseCallback"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(GLFWwindowclosefun cbfun)", },
+       ["glfwSetWindowRefreshCallback"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(GLFWwindowrefreshfun cbfun)", },
+       ["glfwGetVideoModes"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(GLFWvidmode *list, int maxcount)", },
+       ["glfwGetDesktopMode"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(GLFWvidmode *mode)", },
+       ["glfwPollEvents"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(void)", },
+       ["glfwWaitEvents"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(void)", },
+       ["glfwGetKey"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(int key)", },
+       ["glfwGetMouseButton"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(int button)", },
+       ["glfwGetMousePos"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(int *xpos, int *ypos)", },
+       ["glfwSetMousePos"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(int xpos, int ypos)", },
+       ["glfwGetMouseWheel"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(void)", },
+       ["glfwSetMouseWheel"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(int pos)", },
+       ["glfwSetKeyCallback"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(GLFWkeyfun cbfun)", },
+       ["glfwSetCharCallback"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(GLFWcharfun cbfun)", },
+       ["glfwSetMouseButtonCallback"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(GLFWmousebuttonfun cbfun)", },
+       ["glfwSetMousePosCallback"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(GLFWmouseposfun cbfun)", },
+       ["glfwSetMouseWheelCallback"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(GLFWmousewheelfun cbfun)", },
+       ["glfwGetJoystickParam"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(int joy, int param)", },
+       ["glfwGetJoystickPos"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(int joy, float *pos, int numaxes)", },
+       ["glfwGetJoystickButtons"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(int joy, unsigned char *buttons, int numbuttons)", },
+       ["glfwGetTime"] = { type ='function',
+          description = "",
+          returns = "(double)",
+          valuetype = nil,
+          args = "(void)", },
+       ["glfwSetTime"] = { type ='function',
+          description = "",
+          returns = "(void)",
+          valuetype = nil,
+          args = "(double time)", },
+       ["glfwSleep"] = { type ='function',
+          description = "",
+          returns = "(void)",
+          valuetype = nil,
+          args = "(double time)", },
+       ["glfwExtensionSupported"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(const char *extension)", },
+       ["glfwGetProcAddress"] = { type ='function',
+          description = "",
+          returns = "(void*)",
+          valuetype = nil,
+          args = "(const char *procname)", },
+       ["glfwGetGLVersion"] = { type ='function',
+          description = "",
+          returns = "(void)",
+          valuetype = nil,
+          args = "(int *major, int *minor, int *rev)", },
+       ["glfwCreateThread"] = { type ='function',
+          description = "",
+          returns = "(GLFWthread)",
+          valuetype = nil,
+          args = "(GLFWthreadfun fun, void *arg)", },
+       ["glfwDestroyThread"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(GLFWthread ID)", },
+       ["glfwWaitThread"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(GLFWthread ID, int waitmode)", },
+       ["glfwGetThreadID"] = { type ='function',
+          description = "",
+          returns = "(GLFWthread)",
+          valuetype = nil,
+          args = "(void)", },
+       ["glfwCreateMutex"] = { type ='function',
+          description = "",
+          returns = "(GLFWmutex)",
+          valuetype = nil,
+          args = "(void)", },
+       ["glfwDestroyMutex"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(GLFWmutex mutex)", },
+       ["glfwLockMutex"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(GLFWmutex mutex)", },
+       ["glfwUnlockMutex"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(GLFWmutex mutex)", },
+       ["glfwCreateCond"] = { type ='function',
+          description = "",
+          returns = "(GLFWcond)",
+          valuetype = nil,
+          args = "(void)", },
+       ["glfwDestroyCond"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(GLFWcond cond)", },
+       ["glfwWaitCond"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(GLFWcond cond, GLFWmutex mutex, double timeout)", },
+       ["glfwSignalCond"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(GLFWcond cond)", },
+       ["glfwBroadcastCond"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(GLFWcond cond)", },
+       ["glfwGetNumberOfProcessors"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(void)", },
+       ["glfwEnable"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(int token)", },
+       ["glfwDisable"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(int token)", },
+       ["glfwReadImage"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(const char *name, GLFWimage *img, int flags)", },
+       ["glfwReadMemoryImage"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(const void *data, long size, GLFWimage *img, int flags)", },
+       ["glfwFreeImage"] = { type ='function',
+          description = "",
+          returns = "()",
+          valuetype = nil,
+          args = "(GLFWimage *img)", },
+       ["glfwLoadTexture2D"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(const char *name, int flags)", },
+       ["glfwLoadMemoryTexture2D"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(const void *data, long size, int flags)", },
+       ["glfwLoadTextureImage2D"] = { type ='function',
+          description = "",
+          returns = "(int)",
+          valuetype = nil,
+          args = "(GLFWimage *img, int flags)", },
+       ["GLFWvidmode"] = { type ='class',
+          description = "",
+          childs =         {
+          ["Width"] = { type ='value', description = "int", valuetype = nil, },
+        ["Height"] = { type ='value', description = "int", valuetype = nil, },
+        ["RedBits"] = { type ='value', description = "int", valuetype = nil, },
+        ["BlueBits"] = { type ='value', description = "int", valuetype = nil, },
+        ["GreenBits"] = { type ='value', description = "int", valuetype = nil, },
+                }
+         },
+       ["GLFWimage"] = { type ='class',
+          description = "",
+          childs =         {
+          ["Width"] = { type ='value', description = "int", valuetype = nil, },
+        ["Height"] = { type ='value', description = "int", valuetype = nil, },
+        ["Format"] = { type ='value', description = "int", valuetype = nil, },
+        ["BytesPerPixel"] = { type ='value', description = "int", valuetype = nil, },
+                }
+         },
+               }
+  return {
+        glfw = {
+      type = 'lib',
+      description = "GLFW window manager",
+      childs = api,
+    },
+      }
+  

cfg/user-sample.lua

@@ -0,0 +1,35 @@
+--[[-- Rename this file to `user.lua` to get loaded
+
+Configuration files are loaded in the following order
+
+1. <application>\config.lua
+2. cfg\user.lua
+3. ~\.zbs\user.lua
+4. -cfg commandline strings
+
+-- an example of how loaded configuration can be modified from this file
+
+local G = ... -- this now points to the global environment in the script
+local luaspec = G.ide.specs['lua']
+luaspec.exts[2] = "luaz"
+luaspec.keywords[1] = luaspec.keywords[1] .. ' foo'
+
+-- change font size to 12
+editor.fontsize = 12 -- this is mapped to ide.config.editor.fontsize
+filehistorylength = 20 -- this is mapped to ide.config.filehistorylength
+
+-- specify full path to love2d executable; this is only needed
+-- if the game folder and the executable are NOT in the same folder.
+path.love2d = 'd:/lua/love/love'
+
+-- specify full path to moai executable if it's not in one of PATH folders
+path.moai = 'd:/lua/moai/moai'
+
+-- specify full path to lua interpreter if you need to use your own version
+path.lua = 'd:/lua/lua'
+
+-- fix an issue with 0d0d0a line endings in MOAI examples,
+-- which may negatively affect breakpoints during debugging
+editor.iofilter = "0d0d0aFix"
+
+--]]--

cfg/user.lua_for_custom_settings.txt

@@ -1,9 +0,0 @@
---[[--
-
-estrela loads configs in the following order
-
-1. <application>\config.lua
-2. cfg\user.lua
-3. -cfg commandline strings
-
---]]--

interpreters/estrelashell.lua

@@ -7,7 +7,7 @@ return {
     local bottomnotebook = ide.frame.bottomnotebook
     bottomnotebook:SetSelection(1)
 
-    ShellExecuteCode(wfilename)
+    ShellExecuteFile(wfilename)
   end,
   fprojdir = function(self,wfilename)
     return wfilename:GetPath(wx.wxPATH_GET_VOLUME)

interpreters/love2d.lua

@@ -0,0 +1,49 @@
+local love2d
+local win = ide.osname == "Windows"
+
+return {
+  name = "Love2d",
+  description = "Love2d game engine",
+  api = {"baselib", "love2d"},
+  frun = function(self,wfilename,rundebug)
+    love2d = love2d or ide.config.path.love2d -- check if the path is configured
+    if not love2d then
+      local sep = win and ';' or ':'
+      local path = (os.getenv('PATH') or '')..sep
+                 ..(GetPathWithSep(self:fworkdir(wfilename)))..sep
+                 ..(os.getenv('HOME') and GetPathWithSep(os.getenv('HOME'))..'bin' or '')
+      local paths = {}
+      for p in path:gmatch("[^"..sep.."]+") do
+        love2d = love2d or GetFullPathIfExists(p, win and 'love.exe' or 'love')
+        table.insert(paths, p)
+      end
+      if not love2d then
+        DisplayOutput("Can't find love2d executable in any of the following folders: "
+          ..table.concat(paths, ", ").."\n")
+        return
+      end
+    end
+
+    if not GetFullPathIfExists(self:fworkdir(wfilename), 'main.lua') then
+      DisplayOutput("Can't find 'main.lua' file in the current project folder.\n")
+      return
+    end
+
+    if rundebug then DebuggerAttachDefault() end
+
+    local cmd = ('"%s" "%s"%s'):format(love2d,
+      self:fworkdir(wfilename), rundebug and ' -debug' or '')
+    -- CommandLineRun(cmd,wdir,tooutput,nohide,stringcallback,uid,endcallback)
+    return CommandLineRun(cmd,self:fworkdir(wfilename),true,false,nil,nil,
+      function() ide.debugger.pid = nil end)
+  end,
+  fprojdir = function(self,wfilename)
+    return wfilename:GetPath(wx.wxPATH_GET_VOLUME)
+  end,
+  fworkdir = function(self,wfilename)
+    return ide.config.path.projectdir or wfilename:GetPath(wx.wxPATH_GET_VOLUME)
+  end,
+  hasdebugger = true,
+  fattachdebug = function(self) DebuggerAttachDefault() end,
+  scratchextloop = true,
+}

interpreters/lua.lua

@@ -1,22 +0,0 @@
-return {
-  name = "Lua",
-  description = "Commandline Lua interpreter",
-  api = {"wxwidgets","baselib"},
-  frun = function(self,wfilename)
-    local mainpath = ide.editorFilename:gsub("[^/\\]+$","")
-    local filepath = wfilename:GetFullPath()
-    local code = ([[
-      xpcall(function() dofile '%s' end,
-        function(err) print(debug.traceback(err)) end)
-      ]]):format(filepath:gsub("\\","/"))
-    local cmd = '"'..mainpath..'/bin/lua.exe" -e "'..code..'"'
-    CommandLineRun(cmd,self:fworkdir(wfilename),true,false)
-  end,
-  fprojdir = function(self,wfilename)
-    return wfilename:GetPath(wx.wxPATH_GET_VOLUME)
-  end,
-  fworkdir = function (self,wfilename)
-    return ide.config.path.projectdir and ide.config.path.projectdir:len()>0 and
-    ide.config.path.projectdir
-  end,
-}

interpreters/luadeb.lua

@@ -1,33 +1,49 @@
+local exe
+
+local function exePath()
+  local mainpath = ide.editorFilename:gsub("[^/\\]+$","")
+  local macExe = mainpath..'bin/lua.app/Contents/MacOS/lua'
+  return ide.config.path.lua or
+    (ide.osname == "Windows" and mainpath..[[bin\lua]]
+     or (wx.wxFileExists(macExe) and macExe or mainpath..'bin/lua'))
+end
+
 return {
-  name = "Lua Debug",
-  description = "Commandline Lua interpreter",
+  name = "Lua",
+  description = "Lua interpreter with debugger",
   api = {"wxwidgets","baselib"},
   frun = function(self,wfilename,rundebug)
-    local mainpath = string.gsub(ide.editorFilename:gsub("[^/\\]+$",""),"\\","/")
-    local filepath = string.gsub(wfilename:GetFullPath(), "\\","/")
+    exe = exe or exePath()
+    local filepath = wfilename:GetFullPath()
     local script
     if rundebug then
       DebuggerAttachDefault()
-      script = (""..
-        "package.path=package.path..';"..mainpath.."lualibs/?/?.lua';"..
-        "package.cpath=package.cpath..';"..mainpath.."bin/clibs/?.dll';"..
-        "require 'mobdebug'; io.stdout:setvbuf('no'); mobdebug.loop('" .. wx.wxGetHostName().."',"..ide.debugger.portnumber..")")
+      script = rundebug
     else
-      script = ([[dofile '%s']]):format(filepath)
+      -- if running on Windows and can't open the file, this may mean that
+      -- the file path includes unicode characters that need special handling
+      local fh = io.open(filepath, "r")
+      if fh then fh:close() end
+      if ide.osname == 'Windows' and pcall(require, "winapi")
+      and wfilename:FileExists() and not fh then
+        winapi.set_encoding(winapi.CP_UTF8)
+        filepath = winapi.short_path(filepath)
+      end
+
+      script = ('dofile [[%s]]'):format(filepath)
     end
-    local code = ([[xpcall(function() %s end,function(err) print(debug.traceback(err)) end)]]):format(script)
-    local cmd = '"'..mainpath..'bin/lua.exe" -e "'..code..'"'
-    return CommandLineRun(cmd,self:fworkdir(wfilename),true,false)
+    local code = ([[xpcall(function() io.stdout:setvbuf('no'); %s end,function(err) print(debug.traceback(err)) end)]]):format(script)
+    local cmd = '"'..exe..'" -e "'..code..'"'
+    -- CommandLineRun(cmd,wdir,tooutput,nohide,stringcallback,uid,endcallback)
+    return CommandLineRun(cmd,self:fworkdir(wfilename),true,false,nil,nil,
+      function() ide.debugger.pid = nil end)
   end,
   fprojdir = function(self,wfilename)
     return wfilename:GetPath(wx.wxPATH_GET_VOLUME)
   end,
   fworkdir = function (self,wfilename)
-    return ide.config.path.projectdir
-    or wfilename:GetPath(wx.wxPATH_GET_VOLUME)
+    return ide.config.path.projectdir or wfilename:GetPath(wx.wxPATH_GET_VOLUME)
   end,
   hasdebugger = true,
-  fattachdebug = function(self)
-    DebuggerAttachDefault()
-  end,
+  fattachdebug = function(self) DebuggerAttachDefault() end,
 }

interpreters/luxinia.lua

@@ -7,7 +7,7 @@ return {
   name = "Luxinia",
   description = "Luxinia project",
   api = {"luxiniaapi","baselib"},
-  fcmdline = function(self,wfilename)
+  frun = function(self,wfilename,withdebug)
     local projdir = ide.config.path.projectdir
     local endstr = (projdir and projdir:len()>0
       and " -p "..projdir or "")
@@ -18,6 +18,7 @@ return {
     local cmd = 'luxinia.exe --nologo'..endstr
     CommandLineRun(cmd,ide.config.path.luxinia,true,true)
   end,
+  fuid = function(self,wfilename) return "luxinia "..(ide.config.path.projectdir or "") end,
   fprojdir = function(self,wfilename)
     local path = GetPathWithSep(wfilename)
     fname = wx.wxFileName(path)

interpreters/luxinia2.lua

@@ -6,7 +6,7 @@ end
 return {
   name = "Luxinia2",
   description = "Luxinia2",
-  api = {"baselib","cg30","cggl30","glfw3","glewgl","assimp20","luxmath","luxgfx","luxscene","luajit2",},
+  api = {"baselib","cg30","cggl30","glfw","glewgl","assimp20","luxmath","luxgfx","luxscene","luajit2",},
 
   finitclient = function(self)
     if (not CommandLineRunning(self:fuid(wfilename))) then return end
@@ -52,7 +52,7 @@ return {
       local editorDir = string.gsub(ide.editorFilename:gsub("[^/\\]+$",""),"\\","/")
       script = ""..
       "package.path=package.path..';"..editorDir.."lualibs/?/?.lua';"..
-      "require 'mobdebug'; io.stdout:setvbuf('no'); mobdebug.start('" .. wx.wxGetHostName().."',"..ide.debugger.portnumber..")"
+      "io.stdout:setvbuf('no'); require('mobdebug').start('" .. ide.debugger.hostname.."',"..ide.debugger.portnumber..")"
 
       args = args..' -es "'..script..'"'..startargs
     else
@@ -77,6 +77,7 @@ return {
     if not rundebug then
       local client = self:finitclient()
       ShellSupportRemote(client,self:fuid(wfilename))
+      pid = nil
     end
     
     return pid

interpreters/moai.lua

@@ -0,0 +1,85 @@
+local moai
+local win = ide.osname == "Windows"
+
+return {
+  name = "Moai",
+  description = "Moai mobile platform",
+  api = {"baselib"},
+  frun = function(self,wfilename,rundebug)
+    moai = moai or ide.config.path.moai -- check if the path is configured
+    if not moai then
+      local sep = win and ';' or ':'
+      local path = (os.getenv('PATH') or '')..sep
+                 ..(os.getenv('MOAI_BIN') or '')..sep
+                 ..(os.getenv('HOME') and os.getenv('HOME') .. '/bin' or '')
+      local paths = {}
+      for p in path:gmatch("[^"..sep.."]+") do
+        moai = moai or GetFullPathIfExists(p, win and 'moai.exe' or 'moai')
+        table.insert(paths, p)
+      end
+      if not moai then
+        DisplayOutput("Can't find moai executable in any of the folders in PATH or MOAI_BIN: "
+          ..table.concat(paths, ", ").."\n")
+        return
+      end
+    end
+
+    local file
+    local epoints = ide.config.moai and ide.config.moai.entrypoints
+    if epoints then
+      epoints = type(epoints) == 'table' and epoints or {epoints}
+      for _,entry in pairs(epoints) do
+        file = GetFullPathIfExists(self:fworkdir(wfilename), entry)
+        if file then break end
+      end
+      if not file then
+        DisplayOutput("Can't find any of the specified entry points ("
+          ..table.concat(epoints, ", ")
+          ..") in the current project; continuing with the current file...\n")
+      end
+    end
+
+    if rundebug then
+      -- start running the application right away
+      DebuggerAttachDefault({runstart=true, startwith = file})
+      local code = (
+[[xpcall(function() 
+    io.stdout:setvbuf('no')
+    require("mobdebug").moai() -- enable debugging for coroutines
+    %s
+  end, function(err) print(debug.traceback(err)) end)]]):format(rundebug)
+      local tmpfile = wx.wxFileName()
+      tmpfile:AssignTempFileName(".")
+      file = tmpfile:GetFullPath()
+      local f = io.open(file, "w")
+      if not f then
+        DisplayOutput("Can't open temporary file '"..file.."' for writing\n")
+        return 
+      end
+      f:write(code)
+      f:close()
+    end
+
+    file = file or wfilename:GetFullPath()
+
+    -- try to find a config file: (1) MOAI_CONFIG, (2) project directory,
+    -- (3) folder with the current file, (4) folder with moai executable
+    local config = GetFullPathIfExists(os.getenv('MOAI_CONFIG') or self:fworkdir(wfilename), 'config.lua')
+      or GetFullPathIfExists(wfilename:GetPath(wx.wxPATH_GET_VOLUME), 'config.lua')
+      or GetFullPathIfExists(wx.wxFileName(moai):GetPath(wx.wxPATH_GET_VOLUME), 'config.lua')
+    local cmd = config and ('"%s" "%s" "%s"'):format(moai, config, file)
+      or ('"%s" "%s"'):format(moai, file)
+    -- CommandLineRun(cmd,wdir,tooutput,nohide,stringcallback,uid,endcallback)
+    return CommandLineRun(cmd,self:fworkdir(wfilename),true,false,nil,nil,
+      function() ide.debugger.pid = nil if rundebug then wx.wxRemoveFile(file) end end)
+  end,
+  fprojdir = function(self,wfilename)
+    return wfilename:GetPath(wx.wxPATH_GET_VOLUME)
+  end,
+  fworkdir = function(self,wfilename)
+    return ide.config.path.projectdir or wfilename:GetPath(wx.wxPATH_GET_VOLUME)
+  end,
+  hasdebugger = true,
+  fattachdebug = function(self) DebuggerAttachDefault() end,
+  scratchextloop = true,
+}

lualibs/luainspect/ast.lua

@@ -0,0 +1,929 @@
+-- luainspect.ast - Lua Abstract Syntax Tree (AST) and token list operations.
+--
+-- Two main structures are maintained.  A Metalua-style AST represents the
+-- nested syntactic structure obtained from the parse.
+-- A separate linear ordered list of tokens represents the syntactic structure
+-- from the lexing, including line information (character positions only not row/columns),
+-- comments, and keywords, which is originally built from the lineinfo attributes
+-- injected by Metalua into the AST (IMPROVE: it probably would be simpler
+-- to obtain this from the lexer directly rather then inferring it from the parsing).
+-- During AST manipulations, the lineinfo maintained in the AST is ignored
+-- because it was found more difficult to maintain and not in the optimal format.
+--
+-- The contained code deals with
+--   - Building the AST from source.
+--   - Building the tokenlist from the AST lineinfo.
+--   - Querying the AST+tokenlist.
+--   - Modifying the AST+tokenlist (including incremental parsing source -> AST)
+--   - Annotating the AST with navigational info (e.g. parent links) to assist queries.
+--   - Dumping the tokenlist for debugging.
+--
+-- (c) 2010 David Manura, MIT License.
+
+
+--! require 'luainspect.typecheck' (context)
+
+-- boilerplate/utility
+-- LUA_PATH="?.lua;/path/to/metalua/src/compiler/?.lua;/path/to/metalua/src/lib/?.lua"
+-- import modules -- order is important
+require "lexer"
+require "gg"
+require "mlp_lexer"
+require "mlp_misc"
+require "mlp_table"
+require "mlp_meta"
+require "mlp_expr"
+require "mlp_stat"
+--require "mlp_ext"
+_G.mlc = {} -- make gg happy
+-- Metalua:IMPROVE: make above imports simpler
+
+local M = {}
+
+--[=TESTSUITE
+-- utilities
+local ops = {}
+ops['=='] = function(a,b) return a == b end
+local function check(opname, a, b)
+  local op = assert(ops[opname])
+  if not op(a,b) then
+    error("fail == " .. tostring(a) .. " " .. tostring(b))
+  end
+end
+--]=]
+
+-- CATEGORY: debug
+local function DEBUG(...)
+  if LUAINSPECT_DEBUG then
+    print('DEBUG:', ...)
+  end
+end
+
+
+-- Converts character position to row,column position in string src.
+-- Add values are 1-indexed.
+function M.pos_to_linecol(pos, src)
+  local linenum = 1
+  local lasteolpos = 0
+  for eolpos in src:gmatch"()\n" do
+    if eolpos > pos then break end
+    linenum = linenum + 1
+    lasteolpos = eolpos
+  end
+  local colnum = pos - lasteolpos
+  return linenum, colnum
+end
+
+-- Removes any sheband ("#!") line from Lua source string.
+-- CATEGORY: Lua parsing
+function M.remove_shebang(src)
+  local shebang = src:match("^#![^\r\n]*")
+  return shebang and (" "):rep(#shebang) .. src:sub(#shebang+1) or src
+end
+
+
+-- Custom version of loadstring that parses out line number info
+-- CATEGORY: Lua parsing
+function M.loadstring(src)
+  local f, err = loadstring(src, "")
+  if f then
+    return f
+  else
+    err = err:gsub('^%[string ""%]:', "")
+    local linenum = assert(err:match("(%d+):"))
+    local colnum = 0
+    local linenum2 = err:match("^%d+: '[^']+' expected %(to close '[^']+' at line (%d+)")
+    return nil, err, linenum, colnum, linenum2
+  end
+end
+
+
+-- helper for ast_from_string.  Raises on error.
+-- FIX? filename currently ignored in Metalua
+-- CATEGORY: Lua parsing
+local function ast_from_string_helper(src, filename)
+  filename = filename or '(string)'
+  local  lx  = mlp.lexer:newstream (src, filename)
+  local  ast = mlp.chunk(lx)
+  return ast
+end
+
+
+-- Counts number of lines in text.
+-- Warning: the decision of whether to count a trailing new-line in a file
+-- or an empty file as a line is a little subjective.  This function currently
+-- defines the line count as 1 plus the number of new line characters.
+-- CATEGORY: utility/string
+local function linecount(text)
+  local n = 1
+  for _ in text:gmatch'\n' do
+    n = n + 1
+  end
+  return n
+end
+
+
+-- Converts Lua source string to Lua AST (via mlp/gg).
+-- CATEGORY: Lua parsing
+function M.ast_from_string(src, filename)
+  local ok, ast = pcall(ast_from_string_helper, src, filename)
+  if not ok then
+    local err = ast
+    err = err:match('[^\n]*')
+    err = err:gsub("^.-:%s*line", "line")
+        -- mlp.chunk prepending this is undesirable.   error(msg,0) would be better in gg.lua. Reported.
+        -- TODO-Metalua: remove when fixed in Metalua.
+    local linenum, colnum = err:match("line (%d+), char (%d+)")
+    if not linenum then
+      -- Metalua libraries may return "...gg.lua:56: .../mlp_misc.lua:179: End-of-file expected"
+      -- without the normal line/char numbers given things like "if x then end end".  Should be
+      -- fixed probably with gg.parse_error in _chunk in mlp_misc.lua.
+      -- TODO-Metalua: remove when fixed in Metalua.
+      linenum = linecount(src)
+      colnum = 1
+    end
+    local linenum2 = nil
+    return nil, err, linenum, colnum, linenum2
+  else
+    return ast
+  end
+end
+
+
+-- Simple comment parser.  Returns Metalua-style comment.
+-- CATEGORY: Lua lexing
+local function quick_parse_comment(src)
+  local s = src:match"^%-%-([^\n]*)()\n$"
+  if s then return {s, 1, #src, 'short'} end
+  local _, s = src:match(lexer.lexer.patterns.long_comment .. '\r?\n?$')
+  if s then return {s, 1, #src, 'long'} end
+  return nil
+end
+--FIX:check new-line correctness
+--note: currently requiring \n at end of single line comment to avoid
+-- incremental compilation with `--x\nf()` and removing \n from still
+-- recognizing as comment `--x`.
+-- currently allowing \r\n at end of long comment since Metalua includes
+-- it in lineinfo of long comment (FIX:Metalua?)
+
+
+-- Gets length of longest prefix string in both provided strings.
+-- Returns max n such that text1:sub(1,n) == text2:sub(1,n) and n <= max(#text1,#text2)
+-- CATEGORY: string utility
+local function longest_prefix(text1, text2)
+  local nmin = 0
+  local nmax = math.min(#text1, #text2)
+  while nmax > nmin do
+    local nmid = math.ceil((nmin+nmax)/2)
+    if text1:sub(1,nmid) == text2:sub(1,nmid) then
+      nmin = nmid
+    else
+      nmax = nmid-1
+    end
+  end
+  return nmin
+end
+
+
+-- Gets length of longest postfix string in both provided strings.
+-- Returns max n such that text1:sub(-n) == text2:sub(-n) and n <= max(#text1,#text2)
+-- CATEGORY: string utility
+local function longest_postfix(text1, text2)
+  local nmin = 0
+  local nmax = math.min(#text1, #text2)
+  while nmax > nmin do
+    local nmid = math.ceil((nmin+nmax)/2)
+    if text1:sub(-nmid) == text2:sub(-nmid) then --[*]
+      nmin = nmid
+    else
+      nmax = nmid-1
+    end
+  end
+  return nmin
+end  -- differs from longest_prefix only on line [*]
+
+
+
+-- Determines AST node that must be re-evaluated upon changing code string from
+-- `src` to `bsrc`, given previous top_ast/tokenlist/src.
+-- Note: decorates top_ast as side-effect.
+-- If preserve is true, then does not expand AST match even if replacement is invalid.
+-- CATEGORY: AST/tokenlist manipulation
+function M.invalidated_code(top_ast, tokenlist, src, bsrc, preserve)
+  -- Converts posiiton range in src to position range in bsrc.
+  local function range_transform(src_fpos, src_lpos)
+    local src_nlpos = #src - src_lpos
+    local bsrc_fpos = src_fpos
+    local bsrc_lpos = #bsrc - src_nlpos
+    return bsrc_fpos, bsrc_lpos
+  end
+
+  if src == bsrc then return end -- up-to-date
+
+  -- Find range of positions in src that differences correspond to.
+  -- Note: for zero byte range, src_pos2 = src_pos1 - 1.
+  local npre = longest_prefix(src, bsrc)
+  local npost = math.min(#src-npre, longest_postfix(src, bsrc))
+      -- note: min avoids overlap ambiguity
+  local src_fpos, src_lpos = 1 + npre, #src - npost
+
+  -- Find smallest AST node containing src range above.  May also
+  -- be contained in (smaller) comment or whitespace.
+  local match_ast, match_comment, iswhitespace =
+      M.smallest_ast_containing_range(top_ast, tokenlist, src_fpos, src_lpos)
+  DEBUG('invalidate-smallest:', match_ast and (match_ast.tag or 'notag'), match_comment, iswhitespace)
+
+  -- Determine which (ast, comment, or whitespace) to match, and get its pos range in src and bsrc.
+  local srcm_fpos, srcm_lpos, bsrcm_fpos, bsrcm_lpos, mast, mtype
+  if iswhitespace then
+    mast, mtype = nil, 'whitespace'
+    srcm_fpos, srcm_lpos = src_fpos, src_lpos
+  elseif match_comment then
+    mast, mtype = match_comment, 'comment'
+    srcm_fpos, srcm_lpos = match_comment.fpos, match_comment.lpos
+  else
+    mast, mtype = match_ast, 'ast'
+    repeat
+      srcm_fpos, srcm_lpos = M.ast_pos_range(mast, tokenlist)
+      if not srcm_fpos then
+        if mast == top_ast then
+          srcm_fpos, srcm_lpos = 1, #src
+          break
+        else
+          M.ensure_parents_marked(top_ast)
+          mast = mast.parent
+        end
+      end
+    until srcm_fpos
+  end
+  bsrcm_fpos, bsrcm_lpos = range_transform(srcm_fpos, srcm_lpos)
+
+  -- Never expand match if preserve specified.
+  if preserve then
+    return srcm_fpos, srcm_lpos, bsrcm_fpos, bsrcm_lpos, mast, mtype
+  end
+
+  -- Determine if replacement could break parent nodes.
+  local isreplacesafe
+  if mtype == 'whitespace' then
+    if bsrc:sub(bsrcm_fpos, bsrcm_lpos):match'^%s*$' then -- replaced with whitespace
+      if bsrc:sub(bsrcm_fpos-1, bsrcm_lpos+1):match'%s' then -- not eliminating whitespace
+        isreplacesafe = true
+      end
+    end
+  elseif mtype == 'comment' then
+    local m2src = bsrc:sub(bsrcm_fpos, bsrcm_lpos)
+    DEBUG('invalidate-comment[' .. m2src .. ']')
+    if quick_parse_comment(m2src) then  -- replaced with comment
+      isreplacesafe = true
+    end
+  end
+  if isreplacesafe then  -- return on safe replacement
+    return srcm_fpos, srcm_lpos, bsrcm_fpos, bsrcm_lpos, mast, mtype
+  end
+
+  -- Find smallest containing statement block that will compile (or top_ast).
+  while 1 do
+    match_ast = M.get_containing_statementblock(match_ast, top_ast)
+    if match_ast == top_ast then
+      return 1,#src, 1, #bsrc, match_ast, 'statblock'
+         -- entire AST invalidated
+    end
+    local srcm_fpos, srcm_lpos = M.ast_pos_range(match_ast, tokenlist)
+    local bsrcm_fpos, bsrcm_lpos = range_transform(srcm_fpos, srcm_lpos)
+    local msrc = bsrc:sub(bsrcm_fpos, bsrcm_lpos)
+    DEBUG('invalidate-statblock:', match_ast and match_ast.tag, '[' .. msrc .. ']')
+    if loadstring(msrc) then -- compiled
+      return srcm_fpos, srcm_lpos, bsrcm_fpos, bsrcm_lpos, match_ast, 'statblock'
+    end
+    M.ensure_parents_marked(top_ast)
+    match_ast = match_ast.parent
+  end
+end
+
+
+-- Walks AST `ast` in arbitrary order, visiting each node `n`, executing `fdown(n)` (if specified)
+-- when doing down and `fup(n)` (if specified) when going if.
+-- CATEGORY: AST walk
+function M.walk(ast, fdown, fup)
+  assert(type(ast) == 'table')
+  if fdown then fdown(ast) end
+  for _,bast in ipairs(ast) do
+    if type(bast) == 'table' then
+      M.walk(bast, fdown, fup)
+    end
+  end
+  if fup then fup(ast) end
+end
+
+
+-- Replaces contents of table t1 with contents of table t2.
+-- Does not change metatable (if any).
+-- This function is useful for swapping one AST node with another
+-- while preserving any references to the node.
+-- CATEGORY: table utility
+function M.switchtable(t1, t2)
+  for k in pairs(t1) do t1[k] = nil end
+  for k in pairs(t2) do t1[k] = t2[k] end
+end
+
+
+-- Inserts all elements in list bt at index i in list t.
+-- CATEGORY: table utility
+local function tinsertlist(t, i, bt)
+  local oldtlen, delta = #t, i - 1
+  for ti = #t + 1, #t + #bt do t[ti] = false end -- preallocate (avoid holes)
+  for ti = oldtlen, i, -1 do t[ti + #bt] = t[ti] end -- shift
+  for bi = 1, #bt do t[bi + delta] = bt[bi] end -- fill
+end
+--[=[TESTSUITE:
+local function _tinsertlist(t, i, bt)
+  for bi=#bt,1,-1 do table.insert(t, i, bt[bi]) end
+end -- equivalent but MUCH less efficient for large tables
+local function _tinsertlist(t, i, bt)
+  for bi=1,#bt do table.insert(t, i+bi-1, bt[bi]) end
+end -- equivalent but MUCH less efficient for large tables
+local t = {}; tinsertlist(t, 1, {}); assert(table.concat(t)=='')
+local t = {}; tinsertlist(t, 1, {2,3}); assert(table.concat(t)=='23')
+local t = {4}; tinsertlist(t, 1, {2,3}); assert(table.concat(t)=='234')
+local t = {2}; tinsertlist(t, 2, {3,4}); assert(table.concat(t)=='234')
+local t = {4,5}; tinsertlist(t, 1, {2,3}); assert(table.concat(t)=='2345')
+local t = {2,5}; tinsertlist(t, 2, {3,4}); assert(table.concat(t)=='2345')
+local t = {2,3}; tinsertlist(t, 3, {4,5}); assert(table.concat(t)=='2345')
+print 'DONE'
+--]=]
+
+
+
+-- Gets list of keyword positions related to node ast in source src
+-- note: ast must be visible, i.e. have lineinfo (e.g. unlike `Id "self" definition).
+-- Note: includes operators.
+-- Note: Assumes ast Metalua-style lineinfo is valid.
+-- CATEGORY: tokenlist build
+function M.get_keywords(ast, src)
+  local list = {}
+  if not ast.lineinfo then return list end
+  -- examine space between each pair of children i and j.
+  -- special cases: 0 is before first child and #ast+1 is after last child
+
+  -- Put children in lexical order.
+  -- Some binary operations have arguments reversed from lexical order.
+  -- For example, `a > b` becomes `Op{'lt', `Id 'b', `Id 'a'}
+  local oast =
+    (ast.tag == 'Op' and #ast == 3 and ast[2].lineinfo.first[3] > ast[3].lineinfo.first[3])
+    and {ast[1], ast[3], ast[2]} or ast
+
+  local i = 0
+  while i <= #ast do
+    -- j is node following i that has lineinfo
+    local j = i+1; while j < #ast+1 and not oast[j].lineinfo do j=j+1 end
+
+    -- Get position range [fpos,lpos] between subsequent children.
+    local fpos
+    if i == 0 then  -- before first child
+      fpos = ast.lineinfo.first[3]
+    else
+      local last = oast[i].lineinfo.last; local c = last.comments
+      fpos = (c and #c > 0 and c[#c][3] or last[3]) + 1
+    end
+    local lpos
+    if j == #ast+1 then  -- after last child
+      lpos = ast.lineinfo.last[3]
+    else
+      local first = oast[j].lineinfo.first; local c = first.comments
+      --DEBUG('first', ast.tag, first[3], src:sub(first[3], first[3]+3))
+      lpos = (c and #c > 0 and c[1][2] or first[3]) - 1
+    end
+
+    -- Find keyword in range.
+    local spos = fpos
+    repeat
+      local mfpos, tok, mlppos = src:match("^%s*()(%a+)()", spos)
+      if not mfpos then
+        mfpos, tok, mlppos = src:match("^%s*()(%p+)()", spos)
+      end
+      if mfpos then
+        local mlpos = mlppos-1
+        if mlpos > lpos then mlpos = lpos end
+      --DEBUG('look', ast.tag, #ast,i,j,'*', mfpos, tok, mlppos, fpos, lpos, src:sub(fpos, fpos+5))
+        if mlpos >= mfpos then
+          list[#list+1] = mfpos
+          list[#list+1] = mlpos
+        end
+      end
+      spos = mlppos
+    until not spos or spos > lpos
+    -- note: finds single keyword.  in `local function` returns only `local`
+    --DEBUG(i,j ,'test[' .. src:sub(fpos, lpos) .. ']')
+
+    i = j  -- next
+
+    --DESIGN:Lua: comment: string.match accepts a start position but not a stop position
+  end
+  return list
+end
+-- Q:Metalua: does ast.lineinfo[loc].comments imply #ast.lineinfo[loc].comments > 0 ?
+
+
+
+-- Generates ordered list of tokens in top_ast/src.
+-- Note: currently ignores operators and parens.
+-- Note: Modifies ast.
+-- Note: Assumes ast Metalua-style lineinfo is valid.
+-- CATEGORY: AST/tokenlist query
+local isterminal = {Nil=true, Dots=true, True=true, False=true, Number=true, String=true,
+  Dots=true, Id=true}
+local function compare_tokens_(atoken, btoken) return atoken.fpos < btoken.fpos end
+function M.ast_to_tokenlist(top_ast, src)
+  local tokens = {} -- {nbytes=#src}
+  local isseen = {}
+  M.walk(top_ast, function(ast)
+    if isterminal[ast.tag] then -- Extract terminal
+      local token = ast
+      if ast.lineinfo then
+        token.fpos, token.lpos, token.ast = ast.lineinfo.first[3], ast.lineinfo.last[3], ast
+        table.insert(tokens, token)
+      end
+    else -- Extract non-terminal
+      local keywordposlist = M.get_keywords(ast, src)
+      for i=1,#keywordposlist,2 do
+        local fpos, lpos = keywordposlist[i], keywordposlist[i+1]
+        local toksrc = src:sub(fpos, lpos)
+        local token = {tag='Keyword', fpos=fpos, lpos=lpos, ast=ast, toksrc}
+        table.insert(tokens, token)
+      end
+    end
+    -- Extract comments
+    for i=1,2 do
+      local comments = ast.lineinfo and ast.lineinfo[i==1 and 'first' or 'last'].comments
+      if comments then for _, comment in ipairs(comments) do
+        if not isseen[comment] then
+          comment.tag = 'Comment'
+          local token = comment
+          token.fpos, token.lpos, token.ast = comment[2], comment[3], comment
+          table.insert(tokens, token)
+          isseen[comment] = true
+        end
+      end end
+    end
+  end, nil)
+  table.sort(tokens, compare_tokens_)
+  return tokens
+end
+
+
+-- Gets tokenlist range [fidx,lidx] covered by ast.  Returns nil,nil if not found.
+--FIX:PERFORMANCE:this is slow on large files.
+-- CATEGORY: AST/tokenlist query
+function M.ast_idx_range_in_tokenlist(tokenlist, ast)
+  -- Get list of primary nodes under ast.
+  local isold = {}; M.walk(ast, function(ast) isold[ast] = true end)
+  -- Get range.
+  local fidx, lidx
+  for idx=1,#tokenlist do
+    local token = tokenlist[idx]
+    if isold[token.ast] then
+      lidx = idx
+      if not fidx then fidx = idx end
+    end
+  end
+  return fidx, lidx
+end
+
+
+-- Gets index range in tokenlist overlapped by character position range [fpos, lpos].
+-- For example, `do ff() end` with range ` ff() ` would match tokens `ff()`.
+-- Tokens partly inside range are counted, so range `f()` would match tokens `ff()`.
+-- If lidx = fidx - 1, then position range is whitespace between tokens lidx (on left)
+-- and fidx (on right), and this may include token pseudoindices 0 (start of file) and
+-- #tokenlist+1 (end of file).
+-- Note: lpos == fpos - 1 indicates zero-width range between chars lpos and fpos.
+-- CATEGORY: tokenlist query
+function M.tokenlist_idx_range_over_pos_range(tokenlist, fpos, lpos)
+  -- Find first/last indices of tokens overlapped (even partly) by position range.
+  local fidx, lidx
+  for idx=1,#tokenlist do
+    local token = tokenlist[idx]
+    --if (token.fpos >= fpos and token.fpos <= lpos) or (token.lpos >= fpos and token.lpos <= lpos) then -- token overlaps range
+    if fpos <= token.lpos and lpos >= token.fpos then -- range overlaps token (even partially)
+      if not fidx then fidx = idx end
+      lidx = idx
+    end
+  end
+  if not fidx then -- on fail, check between tokens
+    for idx=1,#tokenlist+1 do  -- between idx-1 and idx
+      local tokfpos, toklpos = tokenlist[idx-1] and tokenlist[idx-1].lpos, tokenlist[idx] and tokenlist[idx].fpos
+      if (not tokfpos or fpos > tokfpos) and (not toklpos or lpos < toklpos) then -- range between tokens
+        return idx, idx-1
+      end
+    end
+  end
+  return fidx, lidx
+end
+--[=[TESTSUITE
+local function test(...)
+  return table.concat({M.tokenlist_idx_range_over_pos_range(...)}, ',')
+end
+check('==', test({}, 2, 2), "1,0")  -- no tokens
+check('==', test({{tag='Id', fpos=1, lpos=1}}, 2, 2), "2,1")  -- right of one token
+check('==', test({{tag='Id', fpos=3, lpos=3}}, 2, 2), "1,0")  -- left of one token
+check('==', test({{tag='Id', fpos=3, lpos=4}}, 2, 3), "1,1")  -- left partial overlap one token
+check('==', test({{tag='Id', fpos=3, lpos=4}}, 4, 5), "1,1")  -- right partial overlap one token
+check('==', test({{tag='Id', fpos=3, lpos=6}}, 4, 5), "1,1")  -- partial inner overlap one token
+check('==', test({{tag='Id', fpos=3, lpos=6}}, 3, 6), "1,1")  -- exact overlap one token
+check('==', test({{tag='Id', fpos=4, lpos=5}}, 3, 6), "1,1")  -- extra overlap one token
+check('==', test({{tag='Id', fpos=2, lpos=3}, {tag='Id', fpos=5, lpos=6}}, 4, 4), "2,1")  -- between tokens, " " exact
+check('==', test({{tag='Id', fpos=2, lpos=3}, {tag='Id', fpos=5, lpos=6}}, 4, 3), "2,1")  -- between tokens, "" on left
+check('==', test({{tag='Id', fpos=2, lpos=3}, {tag='Id', fpos=5, lpos=6}}, 5, 4), "2,1")  -- between tokens, "" on right
+check('==', test({{tag='Id', fpos=2, lpos=3}, {tag='Id', fpos=4, lpos=5}}, 4, 3), "2,1")  -- between tokens, "" exact
+--]=]
+
+-- Removes tokens in tokenlist covered by ast.
+-- CATEGORY: tokenlist manipulation
+local function remove_ast_in_tokenlist(tokenlist, ast)
+  local fidx, lidx  = M.ast_idx_range_in_tokenlist(tokenlist, ast)
+  if fidx then  -- note: fidx implies lidx
+    for idx=lidx,fidx,-1 do table.remove(tokenlist, idx) end
+  end
+end
+
+
+-- Inserts tokens from btokenlist into tokenlist.  Preserves sort.
+-- CATEGORY: tokenlist manipulation
+local function insert_tokenlist(tokenlist, btokenlist)
+  local ftoken = btokenlist[1]
+  if ftoken then
+    -- Get index in tokenlist in which to insert tokens in btokenlist.
+    local fidx
+    for idx=1,#tokenlist do
+      if tokenlist[idx].fpos > ftoken.fpos then fidx = idx; break end
+    end
+    fidx = fidx or #tokenlist + 1  -- else append
+
+    -- Insert tokens.
+    tinsertlist(tokenlist, fidx, btokenlist)
+  end
+end
+
+
+-- Get character position range covered by ast in tokenlist.  Returns nil,nil on not found.
+-- CATEGORY: AST/tokenlist query
+function M.ast_pos_range(ast, tokenlist) -- IMPROVE:style: ast_idx_range_in_tokenlist has params reversed
+  local fidx, lidx  = M.ast_idx_range_in_tokenlist(tokenlist, ast)
+  if fidx then
+    return tokenlist[fidx].fpos, tokenlist[lidx].lpos
+  else
+    return nil, nil
+  end
+end
+
+
+-- Gets string representation of AST node.  nil if none.
+-- IMPROVE: what if node is empty block?
+-- CATEGORY: AST/tokenlist query
+function M.ast_to_text(ast, tokenlist, src) -- IMPROVE:style: ast_idx_range_in_tokenlist has params reversed
+  local fpos, lpos = M.ast_pos_range(ast, tokenlist)
+  if fpos then
+    return src:sub(fpos, lpos)
+  else
+    return nil
+  end
+end
+
+
+
+-- Gets smallest AST node in top_ast/tokenlist/src
+-- completely containing position range [pos1, pos2].
+-- careful: "function" is not part of the `Function node.
+-- If range is inside comment, returns comment also.
+-- If range is inside whitespace, then returns true in third return value.
+-- CATEGORY: AST/tokenlist query
+function M.smallest_ast_containing_range(top_ast, tokenlist, pos1, pos2)
+  local f0idx, l0idx = M.tokenlist_idx_range_over_pos_range(tokenlist, pos1, pos2)
+
+  -- Find enclosing AST.
+  M.ensure_parents_marked(top_ast)
+  local fidx, lidx = f0idx, l0idx
+  while tokenlist[fidx] and not tokenlist[fidx].ast.parent do fidx = fidx - 1 end
+  while tokenlist[lidx] and not tokenlist[lidx].ast.parent do lidx = lidx + 1 end
+  -- DEBUG(fidx, lidx, f0idx, l0idx, #tokenlist, pos1, pos2, tokenlist[fidx], tokenlist[lidx])
+  local ast = not (tokenlist[fidx] and tokenlist[lidx]) and top_ast or
+      M.common_ast_parent(tokenlist[fidx].ast, tokenlist[lidx].ast, top_ast)
+  -- DEBUG('m2', tokenlist[fidx], tokenlist[lidx], top_ast, ast, ast and ast.tag)
+  if l0idx == f0idx - 1 then -- whitespace
+    return ast, nil, true
+  elseif l0idx == f0idx and tokenlist[l0idx].tag == 'Comment' then
+    return ast, tokenlist[l0idx], nil
+  else
+    return ast, nil, nil
+  end
+end
+--IMPROVE: handle string edits and maybe others
+
+
+-- Gets smallest statement block containing position pos or
+-- nearest statement block before pos, whichever is smaller, given ast/tokenlist.
+function M.current_statementblock(ast, tokenlist, pos)
+  local fidx,lidx = M.tokenlist_idx_range_over_pos_range(tokenlist, pos, pos)
+  if fidx > lidx then fidx = lidx end -- use nearest backward
+
+  -- Find closest AST node backward
+  while fidx >= 1 and tokenlist[fidx].tag == 'Comment' do fidx=fidx-1 end
+
+  if fidx < 1 then return ast, false end
+  local mast = tokenlist[fidx].ast
+  if not mast then return ast, false end
+  mast = M.get_containing_statementblock(mast, ast)
+  local isafter = false
+  if mast.tag2 ~= 'Block' then
+    local mfidx,mlidx = M.ast_idx_range_in_tokenlist(tokenlist, mast)
+    if pos > mlidx then
+      isafter = true
+    end
+  end
+
+  return mast, isafter
+end
+
+-- Gets index of bast in ast (nil if not found).
+-- CATEGORY: AST query
+function M.ast_idx(ast, bast)
+  for idx=1,#ast do
+    if ast[idx] == bast then return idx end
+  end
+  return nil
+end
+
+
+-- Gets parent of ast and index of ast in parent.
+-- Root node top_ast must also be provided.  Returns nil, nil if ast is root.
+-- Note: may call mark_parents.
+-- CATEGORY: AST query
+function M.ast_parent_idx(top_ast, ast)
+  if ast == top_ast then return nil, nil end
+  M.ensure_parents_marked(top_ast); assert(ast.parent)
+  local idx = M.ast_idx(ast.parent, ast)
+  return ast.parent, idx
+end
+
+
+-- Gets common parent of aast and bast.  Always returns value.
+-- Must provide root top_ast too.
+-- CATEGORY: AST query
+function M.common_ast_parent(aast, bast, top_ast)
+  M.ensure_parents_marked(top_ast)
+  local isparent = {}
+  local tast = bast; repeat isparent[tast] = true; tast = tast.parent until not tast
+  local uast = aast; repeat if isparent[uast] then return uast end; uast = uast.parent until not uast
+  assert(false)
+end
+
+
+-- Replaces old_ast with new_ast/new_tokenlist in top_ast/tokenlist.
+-- Note: assumes new_ast is a block.  assumes old_ast is a statement or block.
+-- CATEGORY: AST/tokenlist
+function M.replace_statements(top_ast, tokenlist, old_ast, new_ast, new_tokenlist)
+  remove_ast_in_tokenlist(tokenlist, old_ast)
+  insert_tokenlist(tokenlist, new_tokenlist)
+  if old_ast == top_ast then -- special case: no parent
+    M.switchtable(old_ast, new_ast) -- note: safe since block is not in tokenlist.
+  else
+    local parent_ast, idx = M.ast_parent_idx(top_ast, old_ast)
+    table.remove(parent_ast, idx)
+    tinsertlist(parent_ast, idx, new_ast)
+  end
+
+  -- fixup annotations
+  for _,bast in ipairs(new_ast) do
+    if top_ast.tag2 then M.mark_tag2(bast, bast.tag == 'Do' and 'StatBlock' or 'Block') end
+    if old_ast.parent then M.mark_parents(bast, old_ast.parent) end
+  end
+end
+
+
+-- Adjusts lineinfo in tokenlist.
+-- All char positions starting at pos1 are shifted by delta number of chars.
+-- CATEGORY: tokenlist
+function M.adjust_lineinfo(tokenlist, pos1, delta)
+  for _,token in ipairs(tokenlist) do
+    if token.fpos >= pos1 then
+       token.fpos = token.fpos + delta
+    end
+    if token.lpos >= pos1 then
+      token.lpos = token.lpos + delta
+    end
+  end
+  --tokenlist.nbytes = tokenlist.nbytes + delta
+end
+
+
+-- For each node n in ast, sets n.parent to parent node of n.
+-- Assumes ast.parent will be parent_ast (may be nil)
+-- CATEGORY: AST query
+function M.mark_parents(ast, parent_ast)
+  ast.parent = parent_ast
+  for _,ast2 in ipairs(ast) do
+    if type(ast2) == 'table' then
+      M.mark_parents(ast2, ast)
+    end
+  end
+end
+
+
+-- Calls mark_parents(ast) if ast not marked.
+-- CATEGORY: AST query
+function M.ensure_parents_marked(ast)
+  if ast[1] and not ast[1].parent then M.mark_parents(ast) end
+end
+
+
+-- For each node n in ast, sets n.tag2 to context string:
+-- 'Block' - node is block
+-- 'Stat' - node is statement
+-- 'StatBlock' - node is statement and block (i.e. `Do)
+-- 'Exp' - node is expression
+-- 'Explist' - node is expression list (or identifier list)
+-- 'Pair' - node is key-value pair in table constructor
+-- note: ast.tag2 will be set to context.
+-- CATEGORY: AST query
+local iscertainstat = {Do=true, Set=true, While=true, Repeat=true, If=true,
+  Fornum=true, Forin=true, Local=true, Localrec=true, Return=true, Break=true}
+function M.mark_tag2(ast, context)
+  context = context or 'Block'
+  ast.tag2 = context
+  for i,bast in ipairs(ast) do
+    if type(bast) == 'table' then
+      local nextcontext
+      if bast.tag == 'Do' then
+        nextcontext = 'StatBlock'
+      elseif iscertainstat[bast.tag] then
+        nextcontext = 'Stat'
+      elseif bast.tag == 'Call' or bast.tag == 'Invoke' then
+        nextcontext = context == 'Block' and 'Stat' or 'Exp'
+        --DESIGN:Metalua: these calls actually contain expression lists,
+        --  but the expression list is not represented as a complete node
+        --  by Metalua (as blocks are in `Do statements)
+      elseif bast.tag == 'Pair' then
+        nextcontext = 'Pair'
+      elseif not bast.tag then
+        if ast.tag == 'Set' or ast.tag == 'Local' or ast.tag == 'Localrec'
+          or ast.tag == 'Forin' and i <= 2
+          or ast.tag == 'Function'  and i == 1
+        then
+          nextcontext = 'Explist'
+        else
+          nextcontext = 'Block'
+        end
+      else
+        nextcontext = 'Exp'
+      end
+      M.mark_tag2(bast, nextcontext)
+    end
+  end
+end
+
+
+-- Gets smallest statement or block containing or being `ast`.
+-- The AST root node `top_ast` must also be provided.
+-- Note: may decorate AST as side-effect (mark_tag2/mark_parents).
+-- top_ast is assumed a block, so this is always successful.
+-- CATEGORY: AST query
+function M.get_containing_statementblock(ast, top_ast)
+  if not top_ast.tag2 then M.mark_tag2(top_ast) end
+  if ast.tag2 == 'Stat' or ast.tag2 == 'StatBlock' or ast.tag2 == 'Block' then
+    return ast
+  else
+    M.ensure_parents_marked(top_ast)
+    return M.get_containing_statementblock(ast.parent, top_ast)
+  end
+end
+
+
+-- Finds smallest statement, block, or comment AST  in ast/tokenlist containing position
+-- range [fpos, lpos].  If allowexpand is true (default nil) and located AST
+-- coincides with position range, then next containing statement is used
+-- instead (this allows multiple calls to further expand the statement selection).
+-- CATEGORY: AST query
+function M.select_statementblockcomment(ast, tokenlist, fpos, lpos, allowexpand)
+--IMPROVE: rename ast to top_ast
+  local match_ast, comment_ast = M.smallest_ast_containing_range(ast, tokenlist, fpos, lpos)
+  local select_ast = comment_ast or M.get_containing_statementblock(match_ast, ast)
+  local nfpos, nlpos = M.ast_pos_range(select_ast, tokenlist)
+  --DEBUG('s', nfpos, nlpos, fpos, lpos, match_ast.tag, select_ast.tag)
+  if allowexpand and fpos == nfpos and lpos == nlpos then
+    if comment_ast then
+      -- Select enclosing statement.
+      select_ast = match_ast
+      nfpos, nlpos = M.ast_pos_range(select_ast, tokenlist)
+    else
+      -- note: multiple times may be needed to expand selection.  For example, in
+      --   `for x=1,2 do f() end` both the statement `f()` and block `f()` have
+      --   the same position range.
+      M.ensure_parents_marked(ast)
+      while select_ast.parent and fpos == nfpos and lpos == nlpos do
+        select_ast = M.get_containing_statementblock(select_ast.parent, ast)
+        nfpos, nlpos = M.ast_pos_range(select_ast, tokenlist)
+      end
+    end
+  end
+  return nfpos, nlpos
+end
+
+
+-- Converts tokenlist to string representation for debugging.
+-- CATEGORY: tokenlist debug
+function M.dump_tokenlist(tokenlist)
+  local ts = {}
+  for i,token in ipairs(tokenlist) do
+    ts[#ts+1] = 'tok.' .. i .. ': [' .. token.fpos .. ',' .. token.lpos .. '] '
+       .. tostring(token[1]) .. ' ' .. tostring(token.ast.tag)
+  end
+  return table.concat(ts, '\n') -- .. 'nbytes=' .. tokenlist.nbytes .. '\n'
+end
+
+
+--FIX:Q: does this handle Unicode ok?
+
+--FIX?:Metalua: fails on string with escape sequence '\/'.  The Reference Manual
+-- doesn't say this sequence is valid though.
+
+--FIX:Metalua: In `local --[[x]] function --[[y]] f() end`,
+--   'x' comment omitted from AST.
+
+--FIX:Metalua: `do --[[x]] end` doesn't generate comments in AST.
+--  `if x then --[[x]] end` and `while 1 do --[[x]] end` generates
+--   comments in first/last of block
+
+--FIX:Metalua: `--[[x]] f() --[[y]]` returns lineinfo around `f()`.
+--  `--[[x]] --[[y]]` returns lineinfo around everything.
+
+--FIX:Metalua: `while 1 do --[[x]] --[[y]] end` returns first > last
+--   lineinfo for contained block
+
+--FIX:Metalua: search for "PATCHED:LuaInspect" in the metalualib folder.
+
+--FIX?:Metalua: loadstring parses "--x" but metalua omits the comment in the AST
+
+--FIX?:Metalua: `local x` is generating `Local{{`Id{x}}, {}}`, which
+--  has no lineinfo on {}.  This is contrary to the Metalua
+--  spec: `Local{ {ident+} {expr+}? }.
+--  Other things like `self` also generate no lineinfo.
+--  The ast2.lineinfo  above avoids this.
+
+--FIX:Metalua: Metalua shouldn't overwrite ipairs/pairs.  Note: Metalua version
+--  doesn't set errorlevel correctly.
+
+--Q:Metalua: Why does `return --[[y]]  z  --[[x]]` have
+--  lineinfo.first.comments, lineinfo.last.comments,
+--  plus lineinfo.comments (which is the same as lineinfo.first.comments) ?
+
+--CAUTION:Metalua: `do  f()   end` returns lineinfo around `do  f()   end`, while
+--  `while 1 do  f()  end` returns lineinfo around `f()` for inner block.
+
+--CAUTION:Metalua: The lineinfo on Metalua comments is inconsistent with other
+--   nodes
+
+--CAUTION:Metalua: lineinfo of table in `f{}` is [3,2], of `f{ x,y }` it's [4,6].
+--  This is inconsistent with `x={}` which is [3,4] and `f""` which is [1,2]
+--  for the string.
+
+--CAUTION:Metalua: only the `function()` form of `Function includes `function`
+--   in lineinfo.  'function' is part of `Localrec and `Set in syntactic sugar form.
+
+
+--[=[TESTSUITE
+-- test longest_prefix/longest_postfix
+local function pr(text1, text2)
+  local lastv
+  local function same(v)
+    assert(not lastv or v == lastv); lastv = v; return v
+  end
+  local function test1(text1, text2) -- test prefix/postfix
+    same(longest_prefix(text1, text2))
+    same(longest_postfix(text1:reverse(), text2:reverse()))
+  end
+  local function test2(text1, text2) -- test swap
+    test1(text1, text2)
+    test1(text2, text1)
+  end
+  for _,extra in ipairs{"", "x", "xy", "xyz"} do -- test extra chars
+    test2(text1, text2..extra)
+    test2(text2, text1..extra)
+  end
+  return lastv
+end
+check('==', pr("",""), 0)
+check('==', pr("a",""), 0)
+check('==', pr("a","a"), 1)
+check('==', pr("ab",""), 0)
+check('==', pr("ab","a"), 1)
+check('==', pr("ab","ab"), 2)
+check('==', pr("abcdefg","abcdefgh"), 7)
+--]=]
+
+--[=[TESTSUITE
+print 'DONE'
+--]=]
+
+
+return M

lualibs/luainspect/compat_env.lua

@@ -0,0 +1,390 @@
+--[[
+
+  compat_env v$(_VERSION) - Lua 5.1/5.2 environment compatibility functions
+
+SYNOPSIS
+
+  -- Get load/loadfile compatibility functions only if using 5.1.
+  local CL = pcall(load, '') and _G or require 'compat_env'
+  local load     = CL.load
+  local loadfile = CL.loadfile
+  
+  -- The following now works in both Lua 5.1 and 5.2:
+  assert(load('return 2*pi', nil, 't', {pi=math.pi}))()
+  assert(loadfile('ex.lua', 't', {print=print}))()
+  
+  -- Get getfenv/setfenv compatibility functions only if using 5.2.
+  local getfenv = _G.getfenv or require 'compat_env'.getfenv
+  local setfenv = _G.setfenv or require 'compat_env'.setfenv
+  local function f() return x end
+  setfenv(f, {x=2})
+  print(x, getfenv(f).x) --> 2, 2
+
+DESCRIPTION
+
+  This module provides Lua 5.1/5.2 environment related compatibility functions.
+  This includes implementations of Lua 5.2 style `load` and `loadfile`
+  for use in Lua 5.1.  It also includes Lua 5.1 style `getfenv` and `setfenv`
+  for use in Lua 5.2.
+ 
+API
+
+  local CL = require 'compat_env'
+  
+  CL.load (ld [, source [, mode [, env] ] ]) --> f [, err]
+
+    This behaves the same as the Lua 5.2 `load` in both
+    Lua 5.1 and 5.2.
+    http://www.lua.org/manual/5.2/manual.html#pdf-load
+    
+  CL.loadfile ([filename [, mode [, env] ] ]) --> f [, err]
+  
+    This behaves the same as the Lua 5.2 `loadfile` in both
+    Lua 5.1 and 5.2.
+    http://www.lua.org/manual/5.2/manual.html#pdf-loadfile
+    
+  CL.getfenv ([f]) --> t
+
+    This is identical to the Lua 5.1 `getfenv` in Lua 5.1.
+    This behaves similar to the Lua 5.1 `getfenv` in Lua 5.2.
+    When a global environment is to be returned, or when `f` is a
+    C function, this returns `_G`  since Lua 5.2 doesn't have
+    (thread) global and C function environments.  This will also
+    return `_G` if the Lua function `f` lacks an `_ENV`
+    upvalue, but it will raise an error if uncertain due to lack of
+    debug info.  It is not normally considered good design to use
+    this function; when possible, use `load` or `loadfile` instead.
+    http://www.lua.org/manual/5.1/manual.html#pdf-getfenv
+    
+  CL.setfenv (f, t)
+  
+    This is identical to the Lua 5.1 `setfenv` in Lua 5.1.
+    This behaves similar to the Lua 5.1 `setfenv` in Lua 5.2.
+    This will do nothing if `f` is a Lua function that
+    lacks an `_ENV` upvalue, but it will raise an error if uncertain
+    due to lack of debug info.  See also Design Notes below.
+    It is not normally considered good design to use
+    this function; when possible, use `load` or `loadfile` instead.
+    http://www.lua.org/manual/5.1/manual.html#pdf-setfenv
+    
+DESIGN NOTES
+
+  This module intends to provide robust and fairly complete reimplementations
+  of the environment related Lua 5.1 and Lua 5.2 functions.
+  No effort is made, however, to simulate rare or difficult to simulate features,
+  such as thread environments, although this is liable to change in the future.
+  Such 5.1 capabilities are discouraged and ideally
+  removed from 5.1 code, thereby allowing your code to work in both 5.1 and 5.2.
+  
+  In Lua 5.2, a `setfenv(f, {})`, where `f` lacks any upvalues, will be silently
+  ignored since there is no `_ENV` in this function to write to, and the
+  environment will have no effect inside the function anyway.  However,
+  this does mean that `getfenv(setfenv(f, t))` does not necessarily equal `t`,
+  which is incompatible with 5.1 code (a possible workaround would be [1]).
+  If `setfenv(f, {})` has an upvalue but no debug info, then this will raise
+  an error to prevent inadvertently executing potentially untrusted code in the
+  global environment.
+  
+  It is not normally considered good design to use `setfenv` and `getfenv`
+  (one reason they were removed in 5.2).  When possible, consider replacing
+  these with `load` or `loadfile`, which are more restrictive and have native
+  implementations in 5.2.
+  
+  This module might be merged into a more general Lua 5.1/5.2 compatibility
+  library (e.g. a full reimplementation of Lua 5.2 `_G`).  However,
+  `load/loadfile/getfenv/setfenv` perhaps are among the more cumbersome
+  functions not to have.
+
+INSTALLATION
+
+  Download compat_env.lua:
+  
+    wget https://raw.github.com/gist/1654007/compat_env.lua
+
+  Copy compat_env.lua into your LUA_PATH.
+  
+  Alternately, unpack, test, and install into LuaRocks:
+  
+     wget https://raw.github.com/gist/1422205/sourceunpack.lua
+     lua sourceunpack.lua compat_env.lua
+     (cd out && luarocks make)
+
+Related work
+
+  http://lua-users.org/wiki/LuaVersionCompatibility
+  https://github.com/stevedonovan/Penlight/blob/master/lua/pl/utils.lua
+    - penlight implementations of getfenv/setfenv
+  http://lua-users.org/lists/lua-l/2010-06/msg00313.html
+    - initial getfenv/setfenv implementation
+    
+References
+
+  [1] http://lua-users.org/lists/lua-l/2010-06/msg00315.html
+
+Copyright
+
+(c) 2012 David Manura.  Licensed under the same terms as Lua 5.1/5.2 (MIT license).
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.
+
+--]]---------------------------------------------------------------------
+
+local M = {_TYPE='module', _NAME='compat_env', _VERSION='0.2.20120124'}
+
+local function check_chunk_type(s, mode)
+  local nmode = mode or 'bt' 
+  local is_binary = s and #s > 0 and s:byte(1) == 27
+  if is_binary and not nmode:match'b' then
+    return nil, ("attempt to load a binary chunk (mode is '%s')"):format(mode)
+  elseif not is_binary and not nmode:match't' then
+    return nil, ("attempt to load a text chunk (mode is '%s')"):format(mode)
+  end
+  return true
+end
+
+local IS_52_LOAD = pcall(load, '')
+if IS_52_LOAD then
+  M.load     = _G.load
+  M.loadfile = _G.loadfile
+else
+  -- 5.2 style `load` implemented in 5.1
+  function M.load(ld, source, mode, env)
+    local f
+    if type(ld) == 'string' then
+      local s = ld
+      local ok, err = check_chunk_type(s, mode); if not ok then return ok, err end
+      local err; f, err = loadstring(s, source); if not f then return f, err end
+    elseif type(ld) == 'function' then
+      local ld2 = ld
+      if (mode or 'bt') ~= 'bt' then
+        local first = ld()
+        local ok, err = check_chunk_type(first, mode); if not ok then return ok, err end
+        ld2 = function()
+          if first then
+            local chunk=first; first=nil; return chunk
+          else return ld() end
+        end
+      end
+      local err; f, err = load(ld2, source); if not f then return f, err end
+    else
+      error(("bad argument #1 to 'load' (function expected, got %s)"):format(type(ld)), 2)
+    end
+    if env then setfenv(f, env) end
+    return f
+  end
+
+  -- 5.2 style `loadfile` implemented in 5.1
+  function M.loadfile(filename, mode, env)
+    if (mode or 'bt') ~= 'bt' then
+      local ioerr
+      local fh, err = io.open(filename, 'rb'); if not fh then return fh, err end
+      local function ld() local chunk; chunk,ioerr = fh:read(4096); return chunk end
+      local f, err = M.load(ld, filename and '@'..filename, mode, env)
+      fh:close()
+      if not f then return f, err end
+      if ioerr then return nil, ioerr end
+      return f
+    else
+      local f, err = loadfile(filename); if not f then return f, err end
+      if env then setfenv(f, env) end
+      return f
+    end
+  end
+end
+
+if _G.setfenv then -- Lua 5.1
+  M.setfenv = _G.setfenv
+  M.getfenv = _G.getfenv
+else -- >= Lua 5.2
+  -- helper function for `getfenv`/`setfenv`
+  local function envlookup(f)
+    local name, val
+    local up = 0
+    local unknown
+    repeat
+      up=up+1; name, val = debug.getupvalue(f, up)
+      if name == '' then unknown = true end
+    until name == '_ENV' or name == nil
+    if name ~= '_ENV' then
+      up = nil
+      if unknown then error("upvalues not readable in Lua 5.2 when debug info missing", 3) end
+    end
+    return (name == '_ENV') and up, val, unknown
+  end
+
+  -- helper function for `getfenv`/`setfenv`
+  local function envhelper(f, name)
+    if type(f) == 'number' then
+      if f < 0 then
+        error(("bad argument #1 to '%s' (level must be non-negative)"):format(name), 3)
+      elseif f < 1 then
+        error("thread environments unsupported in Lua 5.2", 3) --[*]
+      end
+      f = debug.getinfo(f+2, 'f').func
+    elseif type(f) ~= 'function' then
+      error(("bad argument #1 to '%s' (number expected, got %s)"):format(type(name, f)), 2)
+    end
+    return f
+  end
+  -- [*] might simulate with table keyed by coroutine.running()
+  
+  -- 5.1 style `setfenv` implemented in 5.2
+  function M.setfenv(f, t)
+    local f = envhelper(f, 'setfenv')
+    local up, val, unknown = envlookup(f)
+    if up then
+      debug.upvaluejoin(f, up, function() return up end, 1) -- unique upvalue [*]
+      debug.setupvalue(f, up, t)
+    else
+      local what = debug.getinfo(f, 'S').what
+      if what ~= 'Lua' and what ~= 'main' then -- not Lua func
+        error("'setfenv' cannot change environment of given object", 2)
+      end -- else ignore no _ENV upvalue (warning: incompatible with 5.1)
+    end
+  end
+  -- [*] http://lua-users.org/lists/lua-l/2010-06/msg00313.html
+
+  -- 5.1 style `getfenv` implemented in 5.2
+  function M.getfenv(f)
+    if f == 0 or f == nil then return _G end -- simulated behavior
+    local f = envhelper(f, 'setfenv')
+    local up, val = envlookup(f)
+    if not up then return _G end -- simulated behavior [**]
+    return val
+  end
+  -- [**] possible reasons: no _ENV upvalue, C function
+end
+
+
+return M
+
+--[[ FILE rockspec.in
+
+package = 'compat_env'
+version = '$(_VERSION)-1'
+source = {
+  url = 'https://raw.github.com/gist/1654007/$(GITID)/compat_env.lua',
+  --url = 'https://raw.github.com/gist/1654007/compat_env.lua', -- latest raw
+  --url = 'https://gist.github.com/gists/1654007/download',
+  md5 = '$(MD5)'
+}
+description = {
+  summary = 'Lua 5.1/5.2 environment compatibility functions',
+  detailed = [=[
+    Provides Lua 5.1/5.2 environment related compatibility functions.
+    This includes implementations of Lua 5.2 style `load` and `loadfile`
+    for use in Lua 5.1.  It also includes Lua 5.1 style `getfenv` and `setfenv`
+    for use in Lua 5.2.
+  ]=],
+  license = 'MIT/X11',
+  homepage = 'https://gist.github.com/1654007',
+  maintainer = 'David Manura'
+}
+dependencies = {}  -- Lua 5.1 or 5.2
+build = {
+  type = 'builtin',
+  modules = {
+    ['compat_env'] = 'compat_env.lua'
+  }
+}
+
+--]]---------------------------------------------------------------------
+
+--[[ FILE test.lua
+
+-- test.lua - test suite for compat_env module.
+
+local CL = require 'compat_env'
+local load     = CL.load
+local loadfile = CL.loadfile
+local setfenv  = CL.setfenv
+local getfenv  = CL.getfenv
+
+local function checkeq(a, b, e)
+  if a ~= b then error(
+    'not equal ['..tostring(a)..'] ['..tostring(b)..'] ['..tostring(e)..']')
+  end
+end
+local function checkerr(pat, ok, err)
+  assert(not ok, 'checkerr')
+  assert(type(err) == 'string' and err:match(pat), err)
+end
+
+-- test `load`
+checkeq(load('return 2')(), 2)
+checkerr('expected near', load'return 2 2')
+checkerr('text chunk', load('return 2', nil, 'b'))
+checkerr('text chunk', load('', nil, 'b'))
+checkerr('binary chunk', load('\027', nil, 't'))
+checkeq(load('return 2*x',nil,'bt',{x=5})(), 10)
+checkeq(debug.getinfo(load('')).source, '')
+checkeq(debug.getinfo(load('', 'foo')).source, 'foo')
+
+-- test `loadfile`
+local fh = assert(io.open('tmp.lua', 'wb'))
+fh:write('return (...) or x')
+fh:close()
+checkeq(loadfile('tmp.lua')(2), 2)
+checkeq(loadfile('tmp.lua', 't')(2), 2)
+checkerr('text chunk', loadfile('tmp.lua', 'b'))
+checkeq(loadfile('tmp.lua', nil, {x=3})(), 3)
+checkeq(debug.getinfo(loadfile('tmp.lua')).source, '@tmp.lua')
+checkeq(debug.getinfo(loadfile('tmp.lua', 't', {})).source, '@tmp.lua')
+os.remove'tmp.lua'
+
+-- test `setfenv`/`getfenv`
+x = 5
+local a,b=true; local function f(c) if a then return x,b,c end end
+setfenv(f, {x=3})
+checkeq(f(), 3)
+checkeq(getfenv(f).x, 3)
+checkerr('cannot change', pcall(setfenv, string.len, {})) -- C function
+checkeq(getfenv(string.len), _G) -- C function
+local function g()
+  setfenv(1, {x=4})
+  checkeq(getfenv(1).x, 4)
+  return x
+end
+checkeq(g(), 4) -- numeric level
+if _G._VERSION ~= 'Lua 5.1' then
+  checkerr('unsupported', pcall(setfenv, 0, {}))
+end
+checkeq(getfenv(0), _G)
+checkeq(getfenv(), _G) -- no arg
+checkeq(x, 5) -- main unaltered
+setfenv(function()end, {}) -- no upvalues, ignore
+checkeq(getfenv(function()end), _G) -- no upvaluse
+if _G._VERSION ~= 'Lua 5.1' then
+  checkeq(getfenv(setfenv(function()end, {})), _G) -- warning: incompatible with 5.1
+end
+x = nil
+
+print 'OK'
+
+--]]---------------------------------------------------------------------
+
+--[[ FILE CHANGES.txt
+0.2.20120124
+  Renamed module to compat_env (from compat_load)
+  Add getfenv/setfenv functions
+
+0.1.20120121
+  Initial public release
+--]]
+

lualibs/luainspect/dump.lua

@@ -0,0 +1,90 @@
+-- Recursive object dumper, for debugging.
+-- (c) 2010 David Manura, MIT License.
+
+local M = {}
+
+-- My own object dumper.
+-- Intended for debugging, not serialization, with compact formatting.
+-- Robust against recursion.
+-- Renders Metalua table tag fields specially {tag=X, ...} --> "`X{...}".
+-- On first call, only pass parameter o.
+-- CATEGORY: AST debug
+local ignore_keys_ = {lineinfo=true}
+local norecurse_keys_ = {parent=true, ast=true}
+local function dumpstring_key_(k, isseen, newindent)
+  local ks = type(k) == 'string' and k:match'^[%a_][%w_]*$' and k or
+             '[' .. M.dumpstring(k, isseen, newindent) .. ']'
+  return ks
+end
+local function sort_keys_(a, b)
+  if type(a) == 'number' and type(b) == 'number' then
+    return a < b
+  elseif type(a) == 'number' then
+    return false
+  elseif type(b) == 'number' then
+    return true
+  elseif type(a) == 'string' and type(b) == 'string' then
+    return a < b
+  else
+    return tostring(a) < tostring(b) -- arbitrary
+  end
+end
+function M.dumpstring(o, isseen, indent, key)
+  isseen = isseen or {}
+  indent = indent or ''
+
+  if type(o) == 'table' then
+    if isseen[o] or norecurse_keys_[key] then
+      return (type(o.tag) == 'string' and '`' .. o.tag .. ':' or '') .. tostring(o)
+    else isseen[o] = true end -- avoid recursion
+
+    local used = {}
+
+    local tag = o.tag
+    local s = '{'
+    if type(o.tag) == 'string' then
+      s = '`' .. tag .. s; used['tag'] = true
+    end
+    local newindent = indent .. '  '
+
+    local ks = {}; for k in pairs(o) do ks[#ks+1] = k end
+    table.sort(ks, sort_keys_)
+    --for i,k in ipairs(ks) do print ('keys', k) end
+
+    local forcenummultiline
+    for k in pairs(o) do
+       if type(k) == 'number' and type(o[k]) == 'table' then forcenummultiline = true end
+    end
+
+    -- inline elements
+    for _,k in ipairs(ks) do
+      if used[k] then -- skip
+      elseif ignore_keys_[k] then used[k] = true
+      elseif (type(k) ~= 'number' or not forcenummultiline) and
+              type(k) ~= 'table' and (type(o[k]) ~= 'table' or norecurse_keys_[k])
+      then
+        s = s .. dumpstring_key_(k, isseen, newindent) .. '=' .. M.dumpstring(o[k], isseen, newindent, k) .. ', '
+        used[k] = true
+      end
+    end
+
+    -- elements on separate lines
+    local done
+    for _,k in ipairs(ks) do
+      if not used[k] then
+        if not done then s = s .. '\n'; done = true end
+        s = s .. newindent .. dumpstring_key_(k, isseen) .. '=' .. M.dumpstring(o[k], isseen, newindent, k) .. ',\n'
+      end
+    end
+    s = s:gsub(',(%s*)$', '%1')
+    s = s .. (done and indent or '') .. '}'
+    return s
+  elseif type(o) == 'string' then
+    return string.format('%q', o)
+  else
+    return tostring(o)
+  end
+end
+
+return M
+

lualibs/luainspect/globals.lua

@@ -0,0 +1,222 @@
+-- LuaInspect.globals - identifier scope analysis
+-- Locates locals, globals, and their definitions.
+--
+-- (c) D.Manura, 2008-2010, MIT license.
+
+-- based on http://lua-users.org/wiki/DetectingUndefinedVariables
+
+local M = {}
+
+--! require 'luainspect.typecheck' (context)
+
+local LA = require "luainspect.ast"
+
+local function definelocal(scope, name, ast)
+  if scope[name] then
+    scope[name].localmasked = true
+    ast.localmasking = scope[name]
+  end
+  scope[name] = ast
+  if name == '_' then ast.isignore = true end
+end
+
+-- Resolves scoping and usages of variable in AST.
+-- Data Notes:
+--   ast.localdefinition refers to lexically scoped definition of `Id node `ast`.
+--     If ast.localdefinition == ast then ast is a "lexical definition".
+--     If ast.localdefinition == nil, then variable is global.
+--   ast.functionlevel is the number of functions the AST is contained in.
+--     ast.functionlevel is defined iff ast is a lexical definition.
+--   ast.isparam is true iff ast is a lexical definition and a function parameter.
+--   ast.isset is true iff ast is a lexical definition and exists an assignment on it.
+--   ast.isused is true iff ast is a lexical definition and has been referred to.
+--   ast.isignore is true if local variable should be ignored (e.g. typically "_")
+--   ast.localmasking - for a lexical definition, this is set to the lexical definition
+--     this is masking (i.e. same name).  nil if not masking.
+--   ast.localmasked - true iff lexical definition masked by another lexical definition.
+--   ast.isfield is true iff `String node ast is used for field access on object,
+--      e.g. x.y or x['y'].z
+--   ast.previous - For `Index{o,s} or `Invoke{o,s,...}, s.previous == o
+local function traverse(ast, scope, globals, level, functionlevel)
+  scope = scope or {}
+
+  local blockrecurse
+  ast.level = level
+
+  -- operations on walking down the AST
+  if ast.tag == 'Local' then
+    blockrecurse = 1
+    -- note: apply new scope after processing values
+  elseif ast.tag == 'Localrec' then
+    local namelist_ast, valuelist_ast = ast[1], ast[2]
+    for _,value_ast in ipairs(namelist_ast) do
+      assert(value_ast.tag == 'Id')
+      local name = value_ast[1]
+      local parentscope = getmetatable(scope).__index
+      definelocal(parentscope, name, value_ast)
+      value_ast.localdefinition = value_ast
+      value_ast.functionlevel = functionlevel
+      value_ast.level = level+1
+    end
+    blockrecurse = 1
+  elseif ast.tag == 'Id' then
+    local name = ast[1]
+    if scope[name] then
+      ast.localdefinition = scope[name]
+      ast.functionlevel = functionlevel
+      scope[name].isused = true
+    else -- global, do nothing
+    end
+  elseif ast.tag == 'Function' then
+    local paramlist_ast, body_ast = ast[1], ast[2]
+    functionlevel = functionlevel + 1
+    for _,param_ast in ipairs(paramlist_ast) do
+      local name = param_ast[1]
+      assert(param_ast.tag == 'Id' or param_ast.tag == 'Dots')
+      if param_ast.tag == 'Id' then
+        definelocal(scope, name, param_ast)
+        param_ast.localdefinition = param_ast
+        param_ast.functionlevel = functionlevel
+        param_ast.isparam = true
+      end
+      param_ast.level = level+1
+    end
+    blockrecurse = 1
+  elseif ast.tag == 'Set' then
+    local reflist_ast, valuelist_ast = ast[1], ast[2]
+    for _,ref_ast in ipairs(reflist_ast) do
+      if ref_ast.tag == 'Id' then
+        local name = ref_ast[1]
+        if scope[name] then
+          scope[name].isset = true
+        else
+          if not globals[name] then
+            globals[name] = {set=ref_ast}
+          end
+        end
+      end
+      ref_ast.level = level+1
+    end
+    --ENHANCE? We could differentiate assignments to x (which indicates that
+    --  x is not const) and assignments to a member of x (which indicates that
+    --  x is not a pointer to const) and assignments to any nested member of x
+    --  (which indicates that x it not a transitive const).
+  elseif ast.tag == 'Fornum' then
+    blockrecurse = 1
+  elseif ast.tag == 'Forin' then
+    blockrecurse = 1
+  end
+
+  -- recurse (depth-first search down the AST)
+  if ast.tag == 'Repeat' then
+    local block_ast, cond_ast = ast[1], ast[2]
+    local scope = scope
+    for _,stat_ast in ipairs(block_ast) do
+      scope = setmetatable({}, {__index = scope})
+      traverse(stat_ast, scope, globals, level+1, functionlevel)
+    end
+    scope = setmetatable({}, {__index = scope})
+    traverse(cond_ast, scope, globals, level+1, functionlevel)
+  elseif ast.tag == 'Fornum' then
+    local name_ast, block_ast = ast[1], ast[#ast]
+    -- eval value list in current scope
+    for i=2, #ast-1 do traverse(ast[i], scope, globals, level+1, functionlevel) end
+    -- eval body in next scope
+    local name = name_ast[1]
+    definelocal(scope, name, name_ast)
+    name_ast.localdefinition = name_ast
+    name_ast.functionlevel = functionlevel
+    traverse(block_ast, scope, globals, level+1, functionlevel)
+  elseif ast.tag == 'Forin' then
+    local namelist_ast, vallist_ast, block_ast = ast[1], ast[2], ast[3]
+    -- eval value list in current scope
+    traverse(vallist_ast, scope, globals, level+1, functionlevel)
+    -- eval body in next scope
+    for _,name_ast in ipairs(namelist_ast) do
+      local name = name_ast[1]
+      definelocal(scope, name, name_ast)
+      name_ast.localdefinition = name_ast
+      name_ast.functionlevel = functionlevel
+      name_ast.level = level+1
+    end
+    traverse(block_ast, scope, globals, level+1, functionlevel)
+  else -- normal
+    for i,v in ipairs(ast) do
+      if i ~= blockrecurse and type(v) == 'table' then
+        local scope = setmetatable({}, {__index = scope})
+        traverse(v, scope, globals, level+1, functionlevel)
+      end
+    end
+  end
+
+  -- operations on walking up the AST
+  if ast.tag == 'Local' then
+    -- Unlike Localrec, variables come into scope after evaluating values.
+    local namelist_ast, valuelist_ast = ast[1], ast[2]
+    for _,name_ast in ipairs(namelist_ast) do
+      assert(name_ast.tag == 'Id')
+      local name = name_ast[1]
+      local parentscope = getmetatable(scope).__index
+      definelocal(parentscope, name, name_ast)
+      name_ast.localdefinition = name_ast
+      name_ast.functionlevel = functionlevel
+      name_ast.level = level+1
+    end
+  elseif ast.tag == 'Index' then
+    if ast[2].tag == 'String' then
+      ast[2].isfield = true
+      ast[2].previous = ast[1]
+    end
+  elseif ast.tag == 'Invoke' then
+    assert(ast[2].tag == 'String')
+    ast[2].isfield = true
+    ast[2].previous = ast[1]
+  end
+end
+
+function M.globals(ast)
+  -- Default list of defined variables.
+  local scope = setmetatable({}, {})
+  local globals = {}
+  traverse(ast, scope, globals, 1, 1) -- Start check.
+
+  return globals
+end
+
+
+-- Gets locals in scope of statement of block ast.  If isafter is true and ast is statement,
+-- uses scope just after statement ast.
+-- Assumes 'parent' attributes on ast are marked.
+-- Returns table mapping name -> AST local definition.
+function M.variables_in_scope(ast, isafter)
+  local scope = {}
+  local cast = ast
+  while cast.parent do
+    local midx = LA.ast_idx(cast.parent, cast)
+    for idx=1,midx do
+      local bast = cast.parent[idx]
+      if bast.tag == 'Localrec' or bast.tag == 'Local' and (idx < midx or isafter) then
+        local names_ast = bast[1]
+        for bidx=1,#names_ast do
+          local name_ast = names_ast[bidx]
+          local name = name_ast[1]
+          scope[name] = name_ast
+        end
+      elseif cast ~= ast and (bast.tag == 'For' or bast.tag == 'Forin' or bast.tag == 'Function') then
+        local names_ast = bast[1]
+        for bidx=1,#names_ast do
+          local name_ast = names_ast[bidx]
+          if name_ast.tag == 'Id' then  --Q: or maybe `Dots should be included
+            local name = name_ast[1]
+            scope[name] = name_ast
+          end
+        end
+      end
+    end
+    cast = cast.parent
+  end
+  return scope
+end
+
+
+return M

lualibs/luainspect/signatures.lua

@@ -0,0 +1,433 @@
+local M = {}
+
+local T = require "luainspect.types"
+
+-- signatures of known globals
+M.global_signatures = {
+  assert = "assert (v [, message])",
+  collectgarbage = "collectgarbage (opt [, arg])",
+  dofile = "dofile (filename)",
+  error = "error (message [, level])",
+  _G = "(table)",
+  getfenv = "getfenv ([f])",
+  getmetatable = "getmetatable (object)",
+  ipairs = "ipairs (t)",
+  load = "load (func [, chunkname])",
+  loadfile = "loadfile ([filename])",
+  loadstring = "loadstring (string [, chunkname])",
+  next = "next (table [, index])",
+  pairs = "pairs (t)",
+  pcall = "pcall (f, arg1, ...)",
+  print = "print (...)",
+  rawequal = "rawequal (v1, v2)",
+  rawget = "rawget (table, index)",
+  rawset = "rawset (table, index, value)",
+  select = "select (index, ...)",
+  setfenv = "setfenv (f, table)",
+  setmetatable = "setmetatable (table, metatable)",
+  tonumber = "tonumber (e [, base])",
+  tostring = "tostring (e)",
+  type = "type (v)",
+  unpack = "unpack (list [, i [, j]])",
+  _VERSION = "(string)",
+  xpcall = "xpcall (f, err)",
+  module = "module (name [, ...])",
+  require = "require (modname)",
+  coroutine = "(table) coroutine manipulation library",
+  debug = "(table) debug facilities library",
+  io = "(table) I/O library",
+  math = "(table) math functions libary",
+  os = "(table) OS facilities library",
+  package = "(table) package library",
+  string = "(table) string manipulation library",
+  table = "(table) table manipulation library",
+  ["coroutine.create"] = "coroutine.create (f)",
+  ["coroutine.resume"] = "coroutine.resume (co [, val1, ...])",
+  ["coroutine.running"] = "coroutine.running ()",
+  ["coroutine.status"] = "coroutine.status (co)",
+  ["coroutine.wrap"] = "coroutine.wrap (f)",
+  ["coroutine.yield"] = "coroutine.yield (...)",
+  ["debug.debug"] = "debug.debug ()",
+  ["debug.getfenv"] = "debug.getfenv (o)",
+  ["debug.gethook"] = "debug.gethook ([thread])",
+  ["debug.getinfo"] = "debug.getinfo ([thread,] function [, what])",
+  ["debug.getlocal"] = "debug.getlocal ([thread,] level, local)",
+  ["debug.getmetatable"] = "debug.getmetatable (object)",
+  ["debug.getregistry"] = "debug.getregistry ()",
+  ["debug.getupvalue"] = "debug.getupvalue (func, up)",
+  ["debug.setfenv"] = "debug.setfenv (object, table)",
+  ["debug.sethook"] = "debug.sethook ([thread,] hook, mask [, count])",
+  ["debug.setlocal"] = "debug.setlocal ([thread,] level, local, value)",
+  ["debug.setmetatable"] = "debug.setmetatable (object, table)",
+  ["debug.setupvalue"] = "debug.setupvalue (func, up, value)",
+  ["debug.traceback"] = "debug.traceback ([thread,] [message] [, level])",
+  ["io.close"] = "io.close ([file])",
+  ["io.flush"] = "io.flush ()",
+  ["io.input"] = "io.input ([file])",
+  ["io.lines"] = "io.lines ([filename])",
+  ["io.open"] = "io.open (filename [, mode])",
+  ["io.output"] = "io.output ([file])",
+  ["io.popen"] = "io.popen (prog [, mode])",
+  ["io.read"] = "io.read (...)",
+  ["io.tmpfile"] = "io.tmpfile ()",
+  ["io.type"] = "io.type (obj)",
+  ["io.write"] = "io.write (...)",
+  ["math.abs"] = "math.abs (x)",
+  ["math.acos"] = "math.acos (x)",
+  ["math.asin"] = "math.asin (x)",
+  ["math.atan"] = "math.atan (x)",
+  ["math.atan2"] = "math.atan2 (y, x)",
+  ["math.ceil"] = "math.ceil (x)",
+  ["math.cos"] = "math.cos (x)",
+  ["math.cosh"] = "math.cosh (x)",
+  ["math.deg"] = "math.deg (x)",
+  ["math.exp"] = "math.exp (x)",
+  ["math.floor"] = "math.floor (x)",
+  ["math.fmod"] = "math.fmod (x, y)",
+  ["math.frexp"] = "math.frexp (x)",
+  ["math.huge"] = "math.huge",
+  ["math.ldexp"] = "math.ldexp (m, e)",
+  ["math.log"] = "math.log (x)",
+  ["math.log10"] = "math.log10 (x)",
+  ["math.max"] = "math.max (x, ...)",
+  ["math.min"] = "math.min (x, ...)",
+  ["math.modf"] = "math.modf (x)",
+  ["math.pi"] = "math.pi",
+  ["math.pow"] = "math.pow (x, y)",
+  ["math.rad"] = "math.rad (x)",
+  ["math.random"] = "math.random ([m [, n]])",
+  ["math.randomseed"] = "math.randomseed (x)",
+  ["math.sin"] = "math.sin (x)",
+  ["math.sinh"] = "math.sinh (x)",
+  ["math.sqrt"] = "math.sqrt (x)",
+  ["math.tan"] = "math.tan (x)",
+  ["math.tanh"] = "math.tanh (x)",
+  ["os.clock"] = "os.clock ()",
+  ["os.date"] = "os.date ([format [, time]])",
+  ["os.difftime"] = "os.difftime (t2, t1)",
+  ["os.execute"] = "os.execute ([command])",
+  ["os.exit"] = "os.exit ([code])",
+  ["os.getenv"] = "os.getenv (varname)",
+  ["os.remove"] = "os.remove (filename)",
+  ["os.rename"] = "os.rename (oldname, newname)",
+  ["os.setlocale"] = "os.setlocale (locale [, category])",
+  ["os.time"] = "os.time ([table])",
+  ["os.tmpname"] = "os.tmpname ()",
+  ["package.cpath"] = "package.cpath",
+  ["package.loaded"] = "package.loaded",
+  ["package.loaders"] = "package.loaders",
+  ["package.loadlib"] = "package.loadlib (libname, funcname)",
+  ["package.path"] = "package.path",
+  ["package.preload"] = "package.preload",
+  ["package.seeall"] = "package.seeall (module)",
+  ["string.byte"] = "string.byte (s [, i [, j]])",
+  ["string.char"] = "string.char (...)",
+  ["string.dump"] = "string.dump (function)",
+  ["string.find"] = "string.find (s, pattern [, init [, plain]])",
+  ["string.format"] = "string.format (formatstring, ...)",
+  ["string.gmatch"] = "string.gmatch (s, pattern)",
+  ["string.gsub"] = "string.gsub (s, pattern, repl [, n])",
+  ["string.len"] = "string.len (s)",
+  ["string.lower"] = "string.lower (s)",
+  ["string.match"] = "string.match (s, pattern [, init])",
+  ["string.rep"] = "string.rep (s, n)",
+  ["string.reverse"] = "string.reverse (s)",
+  ["string.sub"] = "string.sub (s, i [, j])",
+  ["string.upper"] = "string.upper (s)",
+  ["table.concat"] = "table.concat (table [, sep [, i [, j]]])",
+  ["table.insert"] = "table.insert (table, [pos,] value)",
+  ["table.maxn"] = "table.maxn (table)",
+  ["table.remove"] = "table.remove (table [, pos])",
+  ["table.sort"] = "table.sort (table [, comp])",
+}
+
+-- utility function.  Converts e.g. name 'math.sqrt' to its value.
+local function resolve_global_helper_(name)
+  local o = _G
+  for fieldname in name:gmatch'[^%.]+' do o = o[fieldname] end
+  return o
+end
+local function resolve_global(name)
+  local a, b = pcall(resolve_global_helper_, name)
+  if a then return b else return nil, b end
+end
+
+-- Same as global_signatures but maps value (not name) to signature.
+M.value_signatures = {}
+local isobject = {['function']=true, ['table']=true, ['userdata']=true, ['coroutine']=true}
+for name,sig in pairs(M.global_signatures) do
+  local val, err = resolve_global(name)
+  if isobject[type(val)] then
+    M.value_signatures[val] = sig
+  end
+end
+
+-- min,max argument counts.
+M.argument_counts = {
+  [assert] = {1,2},
+  [collectgarbage] = {1,2},
+  [dofile] = {1},
+  [error] = {1,2},
+  [getfenv or false] = {0,1},
+  [getmetatable] = {1,1},
+  [ipairs] = {1,1},
+  [load] = {1,2},
+  [loadfile] = {0,1},
+  [loadstring] = {1,2},
+  [next] = {1,2},
+  [pairs] = {1,1},
+  [pcall] = {1,math.huge},
+  [print] = {0,math.huge},
+  [rawequal] = {2,2},
+  [rawget] = {2,2},
+  [rawset] = {3,3},
+  [select] = {1, math.huge},
+  [setfenv or false] = {2,2},
+  [setmetatable] = {2,2},
+  [tonumber] = {1,2},
+  [tostring] = {1},
+  [type] = {1},
+  [unpack] = {1,3},
+  [xpcall] = {2,2},
+  [module] = {1,math.huge},
+  [require] = {1,1},
+  [coroutine.create] = {1,1},
+  [coroutine.resume] = {1, math.huge},
+  [coroutine.running] = {0,0},
+  [coroutine.status] = {1,1},
+  [coroutine.wrap] = {1,1},
+  [coroutine.yield] = {0,math.huge},
+  [debug.debug] = {0,0},
+  [debug.getfenv or false] = {1,1},
+  [debug.gethook] = {0,1},
+  [debug.getinfo] = {1,3},
+  [debug.getlocal] = {2,3},
+  [debug.getmetatable] = {1,1},
+  [debug.getregistry] = {0,0},
+  [debug.getupvalue] = {2,2},
+  [debug.setfenv or false] = {2,2},
+  [debug.sethook] = {2,4},
+  [debug.setlocal] = {3,4},
+  [debug.setmetatable] = {2,2},
+  [debug.setupvalue] = {3,3},
+  [debug.traceback] = {0,3},
+  [io.close] = {0,1},
+  [io.flush] = {0,0},
+  [io.input] = {0,1},
+  [io.lines] = {0,1},
+  [io.open] = {1,2},
+  [io.output] = {0,1},
+  [io.popen] = {1,2},
+  [io.read] = {0,math.huge},
+  [io.tmpfile] = {0},
+  [io.type] = {1},
+  [io.write] = {0,math.huge},
+  [math.abs] = {1},
+  [math.acos] = {1},
+  [math.asin] = {1},
+  [math.atan] = {1},
+  [math.atan2] = {2,2},
+  [math.ceil] = {1,1},
+  [math.cos] = {1,1},
+  [math.cosh] = {1,1},
+  [math.deg] = {1,1},
+  [math.exp] = {1,1},
+  [math.floor] = {1,1},
+  [math.fmod] = {2,2},
+  [math.frexp] = {1,1},
+  [math.ldexp] = {2,2},
+  [math.log] = {1,1},
+  [math.log10] = {1,1},
+  [math.max] = {1,math.huge},
+  [math.min] = {1,math.huge},
+  [math.modf] = {1,1},
+  [math.pow] = {2,2},
+  [math.rad] = {1,1},
+  [math.random] = {0,2},
+  [math.randomseed] = {1,1},
+  [math.sin] = {1,1},
+  [math.sinh] = {1,1},
+  [math.sqrt] = {1,1},
+  [math.tan] = {1,1},
+  [math.tanh] = {1,1},
+  [os.clock] = {0,0},
+  [os.date] = {0,2},
+  [os.difftime] = {2,2},
+  [os.execute] = {0,1},
+  [os.exit] = {0,1},
+  [os.getenv] = {1,1},
+  [os.remove] = {1,1},
+  [os.rename] = {2,2},
+  [os.setlocale] = {1,2},
+  [os.time] = {0,1},
+  [os.tmpname] = {0,0},
+  [package.loadlib] = {2,2},
+  [package.seeall] = {1,1},
+  [string.byte] = {1,3},
+  [string.char] = {0,math.huge},
+  [string.dump] = {1,1},
+  [string.find] = {2,4},
+  [string.format] = {1,math.huge},
+  [string.gmatch] = {2,2},
+  [string.gsub] = {3,4},
+  [string.len] = {1,1},
+  [string.lower] = {1,1},
+  [string.match] = {2,3},
+  [string.rep] = {2,2},
+  [string.reverse] = {1,1},
+  [string.sub] = {2,3},
+  [string.upper] = {1,1},
+  [table.concat] = {1,4},
+  [table.insert] = {2,3},
+  [table.maxn] = {1,1},
+  [table.remove] = {1,2},
+  [table.sort] = {1,2},
+  [false] = nil -- trick (relies on potentially undefined behavior)
+}
+
+
+-- functions with zero or nearly zero side-effects, and with deterministic results, that may be evaluated by the analyzer.
+M.safe_function = {
+  [require] = true,
+  [rawequal] = true,
+  [rawget] = true,
+  [require] = true,  -- sort of
+  [select] = true,
+  [tonumber] = true,
+  [tostring] = true,
+  [type] = true,
+  [unpack] = true,
+  [coroutine.create] = true,
+  -- [coroutine.resume]
+  [coroutine.running] = true,
+  [coroutine.status] = true,
+  [coroutine.wrap] = true,
+  --[coroutine.yield]
+  -- [debug.debug]
+  --[debug.getfenv] = true,
+  [debug.gethook] = true,
+  [debug.getinfo] = true,
+  [debug.getlocal] = true,
+  [debug.getmetatable] = true,
+  [debug.getregistry] = true,
+  [debug.getupvalue] = true,
+  -- [debug.setfenv]
+  -- [debug.sethook]
+  -- [debug.setlocal]
+  -- [debug.setmetatable]
+  -- [debug.setupvalue]
+  -- [debug.traceback] = true,
+  [io.type] = true,
+  -- skip all other io.*
+  [math.abs] = true,
+  [math.acos] = true,
+  [math.asin] = true,
+  [math.atan] = true,
+  [math.atan2] = true,
+  [math.ceil] = true,
+  [math.cos] = true,
+  [math.cosh] = true,
+  [math.deg] = true,
+  [math.exp] = true,
+  [math.floor] = true,
+  [math.fmod] = true,
+  [math.frexp] = true,
+  [math.ldexp] = true,
+  [math.log] = true,
+  [math.log10] = true,
+  [math.max] = true,
+  [math.min] = true,
+  [math.modf] = true,
+  [math.pow] = true,
+  [math.rad] = true,
+  --[math.random]
+  --[math.randomseed]
+  [math.sin] = true,
+  [math.sinh] = true,
+  [math.sqrt] = true,
+  [math.tan] = true,
+  [math.tanh] = true,
+  [os.clock] = true, -- safe but non-deterministic
+  [os.date]  = true,-- safe but non-deterministic
+  [os.difftime] = true,
+  --[os.execute]
+  --[os.exit]
+  [os.getenv] = true, -- though depends on environment
+  --[os.remove]
+  --[os.rename]
+  --[os.setlocale]
+  [os.time] = true, -- safe but non-deterministic
+  --[os.tmpname]
+  [string.byte] = true,
+  [string.char] = true,
+  [string.dump] = true,
+  [string.find] = true,
+  [string.format] = true,
+  [string.gmatch] = true,
+  [string.gsub] = true,
+  [string.len] = true,
+  [string.lower] = true,
+  [string.match] = true,
+  [string.rep] = true,
+  [string.reverse] = true,
+  [string.sub] = true,
+  [string.upper] = true,
+  [table.maxn] = true,
+}
+
+M.mock_functions = {}
+
+-- TODO:IMPROVE
+local function mockfunction(func, ...)
+  local inputs = {n=0}
+  local outputs = {n=0}
+  local isoutputs
+  for i=1,select('#', ...) do
+    local v = select(i, ...)
+    if type(v) == 'table' then v = v[1] end
+    if v == 'N' or v == 'I' then v = T.number end
+    if v == '->' then
+      isoutputs = true
+    elseif isoutputs then
+      outputs[#outputs+1] = v; outputs.n = outputs.n + 1
+    else
+      inputs[#inputs+1] = v; inputs.n = inputs.n + 1
+    end
+  end
+  M.mock_functions[func] = {inputs=inputs, outputs=outputs}
+end
+
+
+mockfunction(math.abs, 'N', '->', {'N',0,math.huge})
+mockfunction(math.acos, {'N',-1,1}, '->', {'N',0,math.pi/2})
+mockfunction(math.asin, {'N',-1,1}, '->', {'N',-math.pi/2,math.pi/2})
+mockfunction(math.atan, {'N',-math.huge,math.huge}, '->',
+                      {'N',-math.pi/2,math.pi/2})
+--FIX atan2
+mockfunction(math.ceil, 'N','->','I')
+mockfunction(math.cos, 'N','->',{'N',-1,1})
+mockfunction(math.cosh, 'N','->',{'N',1,math.huge})
+mockfunction(math.deg, 'N','->','N')
+mockfunction(math.exp, 'N','->',{'N',0,math.huge})
+mockfunction(math.floor, 'N','->','I')
+mockfunction(math.fmod, 'N','N','->','N')
+mockfunction(math.frexp, 'N','->',{'N',-1,1},'->','I')
+mockfunction(math.ldexp, {'N','I'},'->','N')
+mockfunction(math.log, {'N',0,math.huge},'->','N')
+mockfunction(math.log10, {'N',0,math.huge},'->','N')
+-- function max(...) print 'NOT IMPL'end
+-- function min(...) print 'NOT IMPL'end
+mockfunction(math.modf, 'N','->','I',{'N',-1,1})
+
+mockfunction(math.pow, 'N','N','->','N') -- improve?
+mockfunction(math.rad, 'N','->','N')
+-- random = function() print 'NOT IMPL' end
+mockfunction(math.randomseed, 'N')
+mockfunction(math.sin, 'N','->',{'N',-1,1})
+mockfunction(math.sinh, 'N','->','N')
+mockfunction(math.sqrt, {'N',0,math.huge},'->',{'N',0,math.huge})
+mockfunction(math.tan, 'N','->','N') -- improve?
+mockfunction(math.tanh, 'N','->',{'N',-1,1})
+
+
+return M

lualibs/luainspect/typecheck.lua

@@ -0,0 +1,40 @@
+-- luainspect.typecheck - Type definitions used to check LuaInspect itself.
+--
+-- (c) 2010 David Manura, MIT License.
+
+local T = require "luainspect.types"
+
+local ast_mt = {__tostring = function(s) return 'AST' end}
+
+return function(context)
+  -- AST type.
+  local ast = T.table {
+    tag = T.string,
+    lineinfo=T.table{first=T.table{comments=T.table{T.table{T.string,T.number,T.number}},T.number,T.number,T.number,T.string},
+                 ast=T.table{comments=T.table{T.table{T.string,T.number,T.number}},T.number,T.number,T.number,T.string}},
+    isfield=T.boolean, tag2=T.string,
+    value=T.universal, valueself=T.number, valuelist=T.table{n=T.number, isvaluepegged=T.boolean},
+    resolvedname=T.string, definedglobal=T.boolean, id=T.number, isparam=T.boolean, isset=T.boolean, isused=T.boolean,
+    isignore=T.boolean,
+    functionlevel=T.number, localmasked=T.boolean, note=T.string, nocollect=T.table{}, isdead=T.boolean}
+     -- FIX: some of these are "boolean or nil" actually
+  ast.localdefinition=ast; ast.localmasking = ast
+  ast.previous = ast; ast.parent = ast
+  ast.seevalue = ast; ast.seenote=ast
+  setmetatable(ast, ast_mt)
+  
+  ast[1] = ast; ast[2] = ast
+  context.apply_value('ast$', ast)
+
+  -- Token type.
+  context.apply_value('token$', T.table{
+    tag=T.string, fpos=T.number, lpos=T.number, keywordid=T.number, ast=ast, [1]=T.string
+  })
+  
+  -- Lua source code string type.
+  context.apply_value('src$', '')
+
+  -- SciTE syler object type.
+  local nf = function()end
+  context.apply_value('^styler$', T.table{SetState=nf, More=nf, Current=nf, Forward=nf, StartStyling=nf, EndStyling=nf, language=T.string})
+end

lualibs/luainspect/types.lua

@@ -0,0 +1,130 @@
+local T = {} -- types
+
+-- istype[o] iff o represents a type (i.e. set of values)
+T.istype = {}
+
+-- iserror[o] iff o represents an error type (created via T.error).
+T.iserror = {}
+
+-- istabletype[o] iff o represents a table type (created by T.table).
+T.istabletype = {}
+
+-- Number type
+T.number = {}
+setmetatable(T.number, T.number)
+function T.number.__tostring(self)
+  return 'number'
+end
+T.istype[T.number] = true
+
+-- String type
+T.string = {}
+setmetatable(T.string, T.string)
+function T.string.__tostring(self)
+  return 'string'
+end
+T.istype[T.string] = true
+
+-- Boolean type
+T.boolean = {}
+setmetatable(T.boolean, T.boolean)
+function T.boolean.__tostring(self)
+  return 'boolean'
+end
+T.istype[T.boolean] = true
+
+-- Table type
+function T.table(t)
+  T.istype[t] = true
+  T.istabletype[t] = true
+  return t
+end
+
+-- Universal type.  This is a superset of all other types.
+T.universal = {}
+setmetatable(T.universal, T.universal)
+function T.universal.__tostring(self)
+  return 'unknown'
+end
+T.istype[T.universal] = true
+
+-- nil type.  Represents `nil` but can be stored in tables.
+T['nil'] = {}
+setmetatable(T['nil'], T['nil'])
+T['nil'].__tostring = function(self)
+  return 'nil'
+end
+T.istype[T['nil']] = true
+
+-- None type.  Represents a non-existent value, in a similar way
+-- that `none` is used differently from `nil` in the Lua C API.
+T.none = {}
+setmetatable(T.none, T.none)
+function T.none.__tostring(self)
+  return 'none'
+end
+T.istype[T.none] = true
+
+-- Error type
+local CError = {}; CError.__index = CError
+function CError.__tostring(self) return "error:" .. tostring(self.value) end
+function T.error(val)
+  local self = setmetatable({value=val}, CError)
+  T.istype[self] = true
+  T.iserror[self] = true
+  return self
+end
+
+
+-- Gets a type that is a superset of the two given types.
+function T.superset_types(a, b)
+  if T.iserror[a] then return a end
+  if T.iserror[b] then return b end
+  if rawequal(a, b) then -- note: including nil == nil
+    return a
+  elseif type(a) == 'string' or a == T.string then
+    if type(b) == 'string' or b == T.string then
+      return T.string
+    else
+      return T.universal
+    end
+  elseif type(a) == 'number' or a == T.number then
+    if type(b) == 'number' or b == T.number then
+      return T.number
+    else
+      return T.universal
+    end
+  elseif type(a) == 'boolean' or a == T.boolean then
+    if type(b) == 'boolean' or b == T.boolean then
+      return T.boolean
+    else
+      return T.universal
+    end
+  else
+    return T.universal -- IMPROVE
+  end
+end
+--[[TESTS:
+assert(T.superset_types(2, 2) == 2)
+assert(T.superset_types(2, 3) == T.number)
+assert(T.superset_types(2, T.number) == T.number)
+assert(T.superset_types(T.number, T.string) == T.universal)
+print 'DONE'
+--]]
+
+-- Determines whether type `o` certainly evaluates to true (true),
+-- certainly evaluates to false (false) or could evaluate to either
+-- true of false ('?').
+function T.boolean_cast(o)
+  if T.iserror[o] then -- special case
+    return '?'
+  elseif o == nil or o == false or o == T['nil'] then -- all subsets of {nil, false}
+    return false
+  elseif o == T.universal or o == T.boolean then -- all supersets of boolean
+    return '?'
+  else -- all subsets of  universal - {nil, false}
+    return true
+  end
+end
+
+return T

lualibs/metalua/gg.lua

@@ -0,0 +1,756 @@
+----------------------------------------------------------------------
+-- Metalua.
+--
+-- Summary: parser generator. Collection of higher order functors,
+--   which allow to build and combine parsers. Relies on a lexer
+--   that supports the same API as the one exposed in mll.lua.
+--
+----------------------------------------------------------------------
+--
+-- Copyright (c) 2006-2008, Fabien Fleutot <metalua@gmail.com>.
+--
+-- This software is released under the MIT Licence, see licence.txt
+-- for details.
+--
+----------------------------------------------------------------------
+
+--------------------------------------------------------------------------------
+--
+-- Exported API:
+--
+-- Parser generators:
+-- * [gg.sequence()]
+-- * [gg.multisequence()]
+-- * [gg.expr()]
+-- * [gg.list()]
+-- * [gg.onkeyword()]
+-- * [gg.optkeyword()]
+--
+-- Other functions: 
+-- * [gg.parse_error()]
+-- * [gg.make_parser()]
+-- * [gg.is_parser()]
+--
+--------------------------------------------------------------------------------
+
+module("gg", package.seeall)
+
+-------------------------------------------------------------------------------
+-- parser metatable, which maps __call to method parse, and adds some
+-- error tracing boilerplate.
+-------------------------------------------------------------------------------
+local parser_metatable = { }
+function parser_metatable.__call (parser, lx, ...)
+   --printf ("Call parser %q of type %q", parser.name or "?", parser.kind)
+   if mlc.metabugs then 
+      return parser:parse (lx, ...) 
+      --local x = parser:parse (lx, ...) 
+      --printf ("Result of parser %q: %s", 
+      --        parser.name or "?",
+      --        _G.table.tostring(x, "nohash", 80))
+      --return x
+   else
+      local li = lx:lineinfo_right() or { "?", "?", "?", "?" }
+      local status, ast = pcall (parser.parse, parser, lx, ...)      
+      if status then return ast else
+         -- Try to replace the gg.lua location, in the error msg, with
+         -- the place where the current parser started handling the
+         -- lexstream.
+         -- Since the error is rethrown, these places are stacked. 
+         error (string.format ("%s\n - (l.%s, c.%s, k.%s) in parser %s", 
+                               ast :strmatch "gg.lua:%d+: (.*)" or ast,
+                               li[1], li[2], li[3], parser.name or parser.kind))
+      end
+   end
+end
+
+-------------------------------------------------------------------------------
+-- Turn a table into a parser, mainly by setting the metatable.
+-------------------------------------------------------------------------------
+function make_parser(kind, p)
+   p.kind = kind
+   if not p.transformers then p.transformers = { } end
+   function p.transformers:add (x)
+      table.insert (self, x)
+   end
+   setmetatable (p, parser_metatable)
+   return p
+end
+
+-------------------------------------------------------------------------------
+-- Return true iff [x] is a parser.
+-- If it's a gg-generated parser, return the name of its kind.
+-------------------------------------------------------------------------------
+function is_parser (x)
+   return type(x)=="function" or getmetatable(x)==parser_metatable and x.kind
+end
+
+-------------------------------------------------------------------------------
+-- Parse a sequence, without applying builder nor transformers
+-------------------------------------------------------------------------------
+local function raw_parse_sequence (lx, p)
+   local r = { }
+   for i=1, #p do
+      e=p[i]
+      if type(e) == "string" then 
+         if not lx:is_keyword (lx:next(), e) then
+            parse_error (lx, "A keyword was expected, probably `%s'.", e) end
+      elseif is_parser (e) then
+         table.insert (r, e (lx)) 
+      else 
+         gg.parse_error (lx,"Sequence `%s': element #%i is neither a string "..
+                         "nor a parser: %s", 
+                         p.name, i, table.tostring(e))
+      end
+   end
+   return r
+end
+
+-------------------------------------------------------------------------------
+-- Parse a multisequence, without applying multisequence transformers.
+-- The sequences are completely parsed.
+-------------------------------------------------------------------------------
+local function raw_parse_multisequence (lx, sequence_table, default)
+   local seq_parser = sequence_table[lx:is_keyword(lx:peek())]
+   if seq_parser  then return seq_parser (lx)
+   elseif default then return default (lx)
+   else return false end
+end
+
+-------------------------------------------------------------------------------
+-- Applies all transformers listed in parser on ast.
+-------------------------------------------------------------------------------
+local function transform (ast, parser, fli, lli)
+   if parser.transformers then
+      for _, t in ipairs (parser.transformers) do ast = t(ast) or ast end
+   end
+   if type(ast) == 'table'then
+      local ali = ast.lineinfo
+      if not ali or ali.first~=fli or ali.last~=lli then
+         ast.lineinfo = { first = fli, last = lli }
+      end
+   end
+   return ast
+end
+
+-------------------------------------------------------------------------------
+-- Generate a tracable parsing error (not implemented yet)
+-------------------------------------------------------------------------------
+function parse_error(lx, fmt, ...)
+   local li = lx:lineinfo_left() or {-1,-1,-1, "<unknown file>"}
+   local msg  = string.format("line %i, char %i: "..fmt, li[1], li[2], ...)   
+   local src = lx.src
+   if li[3]>0 and src then
+      local i, j = li[3], li[3]
+      while src:sub(i,i) ~= '\n' and i>=0    do i=i-1 end
+      while src:sub(j,j) ~= '\n' and j<=#src do j=j+1 end      
+      local srcline = src:sub (i+1, j-1)
+      local idx  = string.rep (" ", li[2]).."^"
+      msg = string.format("%s\n>>> %s\n>>> %s", msg, srcline, idx)
+   end
+   error(msg)
+end
+   
+-------------------------------------------------------------------------------
+--
+-- Sequence parser generator
+--
+-------------------------------------------------------------------------------
+-- Input fields:
+--
+-- * [builder]: how to build an AST out of sequence parts. let [x] be the list
+--   of subparser results (keywords are simply omitted). [builder] can be:
+--    - [nil], in which case the result of parsing is simply [x]
+--    - a string, which is then put as a tag on [x]
+--    - a function, which takes [x] as a parameter and returns an AST.
+--
+-- * [name]: the name of the parser. Used for debug messages
+--
+-- * [transformers]: a list of AST->AST functions, applied in order on ASTs
+--   returned by the parser.
+--
+-- * Table-part entries corresponds to keywords (strings) and subparsers 
+--   (function and callable objects).
+--
+-- After creation, the following fields are added:
+-- * [parse] the parsing function lexer->AST
+-- * [kind] == "sequence"
+-- * [name] is set, if it wasn't in the input.
+--
+-------------------------------------------------------------------------------
+function sequence (p)
+   make_parser ("sequence", p)
+
+   -------------------------------------------------------------------
+   -- Parsing method
+   -------------------------------------------------------------------
+   function p:parse (lx)
+      -- Raw parsing:
+      local fli = lx:lineinfo_right()
+      local seq = raw_parse_sequence (lx, self)
+      local lli = lx:lineinfo_left()
+
+      -- Builder application:
+      local builder, tb = self.builder, type (self.builder)
+      if tb == "string" then seq.tag = builder
+      elseif tb == "function" or builder and builder.__call then seq = builder(seq)
+      elseif builder == nil then -- nothing
+      else error ("Invalid builder of type "..tb.." in sequence") end
+      seq = transform (seq, self, fli, lli)
+      assert (not seq or seq.lineinfo)
+      return seq
+   end
+
+   -------------------------------------------------------------------
+   -- Construction
+   -------------------------------------------------------------------
+   -- Try to build a proper name
+   if p.name then
+      -- don't touch existing name
+   elseif type(p[1])=="string" then -- find name based on 1st keyword
+      if #p==1 then p.name=p[1]
+      elseif type(p[#p])=="string" then
+         p.name = p[1] .. " ... " .. p[#p]
+      else p.name = p[1] .. " ..." end
+   else -- can't find a decent name
+      p.name = "<anonymous>"
+   end
+
+   return p
+end --</sequence>
+
+
+-------------------------------------------------------------------------------
+--
+-- Multiple, keyword-driven, sequence parser generator
+--
+-------------------------------------------------------------------------------
+-- in [p], useful fields are:
+--
+-- * [transformers]: as usual
+--
+-- * [name]: as usual
+--
+-- * Table-part entries must be sequence parsers, or tables which can
+--   be turned into a sequence parser by [gg.sequence]. These
+--   sequences must start with a keyword, and this initial keyword
+--   must be different for each sequence.  The table-part entries will
+--   be removed after [gg.multisequence] returns.
+--
+-- * [default]: the parser to run if the next keyword in the lexer is
+--   none of the registered initial keywords. If there's no default
+--   parser and no suitable initial keyword, the multisequence parser
+--   simply returns [false].
+--
+-- After creation, the following fields are added:
+--
+-- * [parse] the parsing function lexer->AST
+--
+-- * [sequences] the table of sequences, indexed by initial keywords.
+--
+-- * [add] method takes a sequence parser or a config table for
+--   [gg.sequence], and adds/replaces the corresponding sequence
+--   parser. If the keyword was already used, the former sequence is
+--   removed and a warning is issued.
+--
+-- * [get] method returns a sequence by its initial keyword
+--
+-- * [kind] == "multisequence"
+--
+-------------------------------------------------------------------------------
+function multisequence (p)   
+   make_parser ("multisequence", p)
+
+   -------------------------------------------------------------------
+   -- Add a sequence (might be just a config table for [gg.sequence])
+   -------------------------------------------------------------------
+   function p:add (s)
+      -- compile if necessary:
+      local keyword = type(s)=='table' and s[1]
+      if type(s)=='table' and not is_parser(s) then sequence(s) end
+      if is_parser(s)~='sequence' or type(keyword)~='string' then 
+         if self.default then -- two defaults
+            error ("In a multisequence parser, all but one sequences "..
+                   "must start with a keyword")
+         else self.default = s end -- first default
+      elseif self.sequences[keyword] then -- duplicate keyword
+         eprintf (" *** Warning: keyword %q overloaded in multisequence ***",
+                  keyword)
+         self.sequences[keyword] = s
+      else -- newly caught keyword
+         self.sequences[keyword] = s
+      end
+   end -- </multisequence.add>
+
+   -------------------------------------------------------------------
+   -- Get the sequence starting with this keyword. [kw :: string]
+   -------------------------------------------------------------------
+   function p:get (kw) return self.sequences [kw] end
+
+   -------------------------------------------------------------------
+   -- Remove the sequence starting with keyword [kw :: string]
+   -------------------------------------------------------------------
+   function p:del (kw) 
+      if not self.sequences[kw] then 
+         eprintf("*** Warning: trying to delete sequence starting "..
+                 "with %q from a multisequence having no such "..
+                 "entry ***", kw) end
+      local removed = self.sequences[kw]
+      self.sequences[kw] = nil 
+      return removed
+   end
+
+   -------------------------------------------------------------------
+   -- Parsing method
+   -------------------------------------------------------------------
+   function p:parse (lx)
+      local fli = lx:lineinfo_right()
+      local x = raw_parse_multisequence (lx, self.sequences, self.default)
+      local lli = lx:lineinfo_left()
+      return transform (x, self, fli, lli)
+   end
+
+   -------------------------------------------------------------------
+   -- Construction
+   -------------------------------------------------------------------
+   -- Register the sequences passed to the constructor. They're going
+   -- from the array part of the parser to the hash part of field
+   -- [sequences]
+   p.sequences = { }
+   for i=1, #p do p:add (p[i]); p[i] = nil end
+
+   -- FIXME: why is this commented out?
+   --if p.default and not is_parser(p.default) then sequence(p.default) end
+   return p
+end --</multisequence>
+
+
+-------------------------------------------------------------------------------
+--
+-- Expression parser generator
+--
+-------------------------------------------------------------------------------
+--
+-- Expression configuration relies on three tables: [prefix], [infix]
+-- and [suffix]. Moreover, the primary parser can be replaced by a
+-- table: in this case the [primary] table will be passed to
+-- [gg.multisequence] to create a parser.
+--
+-- Each of these tables is a modified multisequence parser: the
+-- differences with respect to regular multisequence config tables are:
+--
+-- * the builder takes specific parameters:
+--   - for [prefix], it takes the result of the prefix sequence parser,
+--     and the prefixed expression
+--   - for [infix], it takes the left-hand-side expression, the results 
+--     of the infix sequence parser, and the right-hand-side expression.
+--   - for [suffix], it takes the suffixed expression, and theresult 
+--     of the suffix sequence parser.
+--
+-- * the default field is a list, with parameters:
+--   - [parser] the raw parsing function
+--   - [transformers], as usual
+--   - [prec], the operator's precedence
+--   - [assoc] for [infix] table, the operator's associativity, which
+--     can be "left", "right" or "flat" (default to left)
+--
+-- In [p], useful fields are:
+-- * [transformers]: as usual
+-- * [name]: as usual
+-- * [primary]: the atomic expression parser, or a multisequence config 
+--   table (mandatory)
+-- * [prefix]:  prefix  operators config table, see above.
+-- * [infix]:   infix   operators config table, see above.
+-- * [suffix]: suffix operators config table, see above.
+--
+-- After creation, these fields are added:
+-- * [kind] == "expr"
+-- * [parse] as usual
+-- * each table is turned into a multisequence, and therefore has an 
+--   [add] method
+--
+-------------------------------------------------------------------------------
+function expr (p)
+   make_parser ("expr", p)
+
+   -------------------------------------------------------------------
+   -- parser method.
+   -- In addition to the lexer, it takes an optional precedence:
+   -- it won't read expressions whose precedence is lower or equal
+   -- to [prec].
+   -------------------------------------------------------------------
+   function p:parse (lx, prec)
+      prec = prec or 0
+
+      ------------------------------------------------------
+      -- Extract the right parser and the corresponding
+      -- options table, for (pre|in|suff)fix operators.
+      -- Options include prec, assoc, transformers.
+      ------------------------------------------------------
+      local function get_parser_info (tab)
+         local p2 = tab:get (lx:is_keyword (lx:peek()))
+         if p2 then -- keyword-based sequence found
+            local function parser(lx) return raw_parse_sequence(lx, p2) end
+            return parser, p2
+         else -- Got to use the default parser
+            local d = tab.default
+            if d then return d.parse or d.parser, d
+            else return false, false end
+         end
+      end
+
+      ------------------------------------------------------
+      -- Look for a prefix sequence. Multiple prefixes are
+      -- handled through the recursive [p.parse] call.
+      -- Notice the double-transform: one for the primary
+      -- expr, and one for the one with the prefix op.
+      ------------------------------------------------------
+      local function handle_prefix ()
+         local fli = lx:lineinfo_right()
+         local p2_func, p2 = get_parser_info (self.prefix)
+         local op = p2_func and p2_func (lx)
+         if op then -- Keyword-based sequence found
+            local ili = lx:lineinfo_right() -- Intermediate LineInfo
+            local e = p2.builder (op, self:parse (lx, p2.prec))
+            local lli = lx:lineinfo_left()
+            return transform (transform (e, p2, ili, lli), self, fli, lli)
+         else -- No prefix found, get a primary expression         
+            local e = self.primary(lx)
+            local lli = lx:lineinfo_left()
+            return transform (e, self, fli, lli)
+         end
+      end --</expr.parse.handle_prefix>
+
+      ------------------------------------------------------
+      -- Look for an infix sequence+right-hand-side operand.
+      -- Return the whole binary expression result,
+      -- or false if no operator was found.
+      ------------------------------------------------------
+      local function handle_infix (e)
+         local p2_func, p2 = get_parser_info (self.infix)
+         if not p2 then return false end
+
+         -----------------------------------------
+         -- Handle flattening operators: gather all operands
+         -- of the series in [list]; when a different operator 
+         -- is found, stop, build from [list], [transform] and
+         -- return.
+         -----------------------------------------
+         if (not p2.prec or p2.prec>prec) and p2.assoc=="flat" then
+            local fli = lx:lineinfo_right()
+            local pflat, list = p2, { e }
+            repeat
+               local op = p2_func(lx)
+               if not op then break end
+               table.insert (list, self:parse (lx, p2.prec))
+               local _ -- We only care about checking that p2==pflat
+               _, p2 = get_parser_info (self.infix)
+            until p2 ~= pflat
+            local e2 = pflat.builder (list)
+            local lli = lx:lineinfo_left()
+            return transform (transform (e2, pflat, fli, lli), self, fli, lli)
+ 
+         -----------------------------------------
+         -- Handle regular infix operators: [e] the LHS is known,
+         -- just gather the operator and [e2] the RHS.
+         -- Result goes in [e3].
+         -----------------------------------------
+         elseif p2.prec and p2.prec>prec or 
+                p2.prec==prec and p2.assoc=="right" then
+            local fli = e.lineinfo.first -- lx:lineinfo_right()
+            local op = p2_func(lx)
+            if not op then return false end
+            local e2 = self:parse (lx, p2.prec)
+            local e3 = p2.builder (e, op, e2)
+            local lli = lx:lineinfo_left()
+            return transform (transform (e3, p2, fli, lli), self, fli, lli)
+
+         -----------------------------------------
+         -- Check for non-associative operators, and complain if applicable. 
+         -----------------------------------------
+         elseif p2.assoc=="none" and p2.prec==prec then
+            parse_error (lx, "non-associative operator!")
+
+         -----------------------------------------
+         -- No infix operator suitable at that precedence
+         -----------------------------------------
+         else return false end
+
+      end --</expr.parse.handle_infix>
+
+      ------------------------------------------------------
+      -- Look for a suffix sequence.
+      -- Return the result of suffix operator on [e],
+      -- or false if no operator was found.
+      ------------------------------------------------------
+      local function handle_suffix (e)
+         -- FIXME bad fli, must take e.lineinfo.first
+         local p2_func, p2 = get_parser_info (self.suffix)
+         if not p2 then return false end
+         if not p2.prec or p2.prec>=prec then
+            --local fli = lx:lineinfo_right()
+            local fli = e.lineinfo.first
+            local op = p2_func(lx)
+            if not op then return false end
+            local lli = lx:lineinfo_left()
+            e = p2.builder (e, op)
+            e = transform (transform (e, p2, fli, lli), self, fli, lli)
+            return e
+         end
+         return false
+      end --</expr.parse.handle_suffix>
+
+      ------------------------------------------------------
+      -- Parser body: read suffix and (infix+operand) 
+      -- extensions as long as we're able to fetch more at
+      -- this precedence level.
+      ------------------------------------------------------
+      local e = handle_prefix()
+      repeat
+         local x = handle_suffix (e); e = x or e
+         local y = handle_infix   (e); e = y or e
+      until not (x or y)
+
+      -- No transform: it already happened in operators handling
+      return e
+   end --</expr.parse>
+
+   -------------------------------------------------------------------
+   -- Construction
+   -------------------------------------------------------------------
+   if not p.primary then p.primary=p[1]; p[1]=nil end
+   for _, t in ipairs{ "primary", "prefix", "infix", "suffix" } do
+      if not p[t] then p[t] = { } end
+      if not is_parser(p[t]) then multisequence(p[t]) end
+   end
+   function p:add(...) return self.primary:add(...) end
+   return p
+end --</expr>
+
+
+-------------------------------------------------------------------------------
+--
+-- List parser generator
+--
+-------------------------------------------------------------------------------
+-- In [p], the following fields can be provided in input:
+--
+-- * [builder]: takes list of subparser results, returns AST
+-- * [transformers]: as usual
+-- * [name]: as usual
+--
+-- * [terminators]: list of strings representing the keywords which
+--   might mark the end of the list. When non-empty, the list is
+--   allowed to be empty. A string is treated as a single-element
+--   table, whose element is that string, e.g. ["do"] is the same as
+--   [{"do"}].
+--
+-- * [separators]: list of strings representing the keywords which can
+--   separate elements of the list. When non-empty, one of these
+--   keyword has to be found between each element. Lack of a separator
+--   indicates the end of the list. A string is treated as a
+--   single-element table, whose element is that string, e.g. ["do"]
+--   is the same as [{"do"}]. If [terminators] is empty/nil, then
+--   [separators] has to be non-empty.
+--
+-- After creation, the following fields are added:
+-- * [parse] the parsing function lexer->AST
+-- * [kind] == "list"
+--
+-------------------------------------------------------------------------------
+function list (p)
+   make_parser ("list", p)
+
+   -------------------------------------------------------------------
+   -- Parsing method
+   -------------------------------------------------------------------
+   function p:parse (lx)
+
+      ------------------------------------------------------
+      -- Used to quickly check whether there's a terminator 
+      -- or a separator immediately ahead
+      ------------------------------------------------------
+      local function peek_is_in (keywords) 
+         return keywords and lx:is_keyword(lx:peek(), unpack(keywords)) end
+
+      local x = { }
+      local fli = lx:lineinfo_right()
+
+      -- if there's a terminator to start with, don't bother trying
+      if not peek_is_in (self.terminators) then 
+         repeat table.insert (x, self.primary (lx)) -- read one element
+         until
+            -- First reason to stop: There's a separator list specified,
+            -- and next token isn't one. Otherwise, consume it with [lx:next()]
+            self.separators and not(peek_is_in (self.separators) and lx:next()) or
+            -- Other reason to stop: terminator token ahead
+            peek_is_in (self.terminators) or
+            -- Last reason: end of file reached
+            lx:peek().tag=="Eof"
+      end
+
+      local lli = lx:lineinfo_left()
+      
+      -- Apply the builder. It can be a string, or a callable value, 
+      -- or simply nothing.
+      local b = self.builder
+      if b then
+         if type(b)=="string" then x.tag = b -- b is a string, use it as a tag
+         elseif type(b)=="function" then x=b(x)
+         else
+            local bmt = getmetatable(b)
+            if bmt and bmt.__call then x=b(x) end
+         end
+      end
+      return transform (x, self, fli, lli)
+   end --</list.parse>
+
+   -------------------------------------------------------------------
+   -- Construction
+   -------------------------------------------------------------------
+   if not p.primary then p.primary = p[1]; p[1] = nil end
+   if type(p.terminators) == "string" then p.terminators = { p.terminators }
+   elseif p.terminators and #p.terminators == 0 then p.terminators = nil end
+   if type(p.separators) == "string" then p.separators = { p.separators }
+   elseif p.separators and #p.separators == 0 then p.separators = nil end
+
+   return p
+end --</list>
+
+
+-------------------------------------------------------------------------------
+--
+-- Keyword-conditionned parser generator
+--
+-------------------------------------------------------------------------------
+-- 
+-- Only apply a parser if a given keyword is found. The result of
+-- [gg.onkeyword] parser is the result of the subparser (modulo
+-- [transformers] applications).
+--
+-- lineinfo: the keyword is *not* included in the boundaries of the
+-- resulting lineinfo. A review of all usages of gg.onkeyword() in the
+-- implementation of metalua has shown that it was the appropriate choice
+-- in every case.
+--
+-- Input fields:
+--
+-- * [name]: as usual
+--
+-- * [transformers]: as usual
+--
+-- * [peek]: if non-nil, the conditionning keyword is left in the lexeme
+--   stream instead of being consumed.
+--
+-- * [primary]: the subparser. 
+--
+-- * [keywords]: list of strings representing triggering keywords.
+--
+-- * Table-part entries can contain strings, and/or exactly one parser.
+--   Strings are put in [keywords], and the parser is put in [primary].
+--
+-- After the call, the following fields will be set:
+--   
+-- * [parse] the parsing method
+-- * [kind] == "onkeyword"
+-- * [primary]
+-- * [keywords]
+--
+-------------------------------------------------------------------------------
+function onkeyword (p)
+   make_parser ("onkeyword", p)
+
+   -------------------------------------------------------------------
+   -- Parsing method
+   -------------------------------------------------------------------
+   function p:parse(lx)
+      if lx:is_keyword (lx:peek(), unpack(self.keywords)) then
+         --local fli = lx:lineinfo_right()
+         if not self.peek then lx:next() end
+         local content = self.primary (lx)
+         --local lli = lx:lineinfo_left()
+         local fli, lli = content.lineinfo.first, content.lineinfo.last
+         return transform (content, p, fli, lli)
+      else return false end
+   end
+
+   -------------------------------------------------------------------
+   -- Construction
+   -------------------------------------------------------------------
+   if not p.keywords then p.keywords = { } end
+   for _, x in ipairs(p) do
+      if type(x)=="string" then table.insert (p.keywords, x)
+      else assert (not p.primary and is_parser (x)); p.primary = x end
+   end
+   if not next (p.keywords) then 
+      eprintf("Warning, no keyword to trigger gg.onkeyword") end
+   assert (p.primary, 'no primary parser in gg.onkeyword')
+   return p
+end --</onkeyword>
+
+
+-------------------------------------------------------------------------------
+--
+-- Optional keyword consummer pseudo-parser generator
+--
+-------------------------------------------------------------------------------
+--
+-- This doesn't return a real parser, just a function. That function parses
+-- one of the keywords passed as parameters, and returns it. It returns 
+-- [false] if no matching keyword is found.
+--
+-- Notice that tokens returned by lexer already carry lineinfo, therefore
+-- there's no need to add them, as done usually through transform() calls.
+-------------------------------------------------------------------------------
+function optkeyword (...)
+   local args = {...}
+   if type (args[1]) == "table" then 
+      assert (#args == 1)
+      args = args[1]
+   end
+   for _, v in ipairs(args) do assert (type(v)=="string") end
+   return function (lx)
+      local x = lx:is_keyword (lx:peek(), unpack (args))
+      if x then lx:next(); return x
+      else return false end
+   end
+end
+
+
+-------------------------------------------------------------------------------
+--
+-- Run a parser with a special lexer
+--
+-------------------------------------------------------------------------------
+--
+-- This doesn't return a real parser, just a function.
+-- First argument is the lexer class to be used with the parser,
+-- 2nd is the parser itself.
+-- The resulting parser returns whatever the argument parser does.
+--
+-------------------------------------------------------------------------------
+function with_lexer(new_lexer, parser)
+
+   -------------------------------------------------------------------
+   -- Most gg functions take their parameters in a table, so it's 
+   -- better to silently accept when with_lexer{ } is called with
+   -- its arguments in a list:
+   -------------------------------------------------------------------
+   if not parser and #new_lexer==2 and type(new_lexer[1])=='table' then
+      return with_lexer(unpack(new_lexer))
+   end
+
+   -------------------------------------------------------------------
+   -- Save the current lexer, switch it for the new one, run the parser,
+   -- restore the previous lexer, even if the parser caused an error.
+   -------------------------------------------------------------------
+   return function (lx)
+      local old_lexer = getmetatable(lx)
+      lx:sync()
+      setmetatable(lx, new_lexer)
+      local status, result = pcall(parser, lx)
+      lx:sync()
+      setmetatable(lx, old_lexer)
+      if status then return result else error(result) end
+   end
+end

lualibs/metalua/ldump.lua

@@ -0,0 +1,441 @@
+----------------------------------------------------------------------
+--
+-- WARNING! You're entering a hackish area, proceed at your own risks!
+--
+-- This code results from the borrowing, then ruthless abuse, of
+-- Yueliang's implementation of Lua 5.0 compiler. I claim
+-- responsibility for all of the ugly, dirty stuff that you might spot
+-- in it.
+--
+-- Eventually, this code will be rewritten, either in Lua or more
+-- probably in C. Meanwhile, if you're interested into digging
+-- metalua's sources, this is not the best part to invest your time
+-- on.
+--
+-- End of warning.
+--
+----------------------------------------------------------------------
+
+--[[--------------------------------------------------------------------
+
+  $Id$
+
+  ldump.lua
+  Save bytecodes in Lua
+  This file is part of Yueliang.
+
+  Copyright (c) 2005 Kein-Hong Man <khman@users.sf.net>
+  The COPYRIGHT file describes the conditions
+  under which this software may be distributed.
+
+------------------------------------------------------------------------
+
+  [FF] Slightly modified, mainly to produce Lua 5.1 bytecode.
+
+----------------------------------------------------------------------]]
+
+--[[--------------------------------------------------------------------
+-- Notes:
+-- * LUA_NUMBER (double), byte order (little endian) and some other
+--   header values hard-coded; see other notes below...
+-- * One significant difference is that instructions are still in table
+--   form (with OP/A/B/C/Bx fields) and luaP:Instruction() is needed to
+--   convert them into 4-char strings
+-- * Deleted:
+--   luaU:DumpVector: folded into DumpLines, DumpCode
+-- * Added:
+--   luaU:endianness() (from lundump.c)
+--   luaU:make_setS: create a chunk writer that writes to a string
+--   luaU:make_setF: create a chunk writer that writes to a file
+--     (lua.h contains a typedef for a Chunkwriter pointer, and
+--      a Lua-based implementation exists, writer() in lstrlib.c)
+--   luaU:from_double(x): encode double value for writing
+--   luaU:from_int(x): encode integer value for writing
+--     (error checking is limited for these conversion functions)
+--     (double conversion does not support denormals or NaNs)
+--   luaU:ttype(o) (from lobject.h)
+----------------------------------------------------------------------]]
+
+module("bytecode", package.seeall)
+
+format = { }
+format.header = string.dump(function()end):sub(1, 12)
+format.little_endian, format.int_size, 
+format.size_t_size,   format.instr_size, 
+format.number_size,   format.integral = format.header:byte(7, 12)
+format.little_endian = format.little_endian~=0
+format.integral      = format.integral     ~=0
+
+assert(format.integral or format.number_size==8, "Number format not supported by dumper")
+assert(format.little_endian, "Big endian architectures not supported by dumper")
+
+--requires luaP
+luaU = {}
+
+-- constants used by dumper
+luaU.LUA_TNIL     = 0
+luaU.LUA_TBOOLEAN = 1
+luaU.LUA_TNUMBER  = 3 -- (all in lua.h)
+luaU.LUA_TSTRING  = 4
+luaU.LUA_TNONE   = -1
+
+-- definitions for headers of binary files
+--luaU.LUA_SIGNATURE = "\27Lua"   -- binary files start with "<esc>Lua"
+--luaU.VERSION = 81               -- 0x50; last format change was in 5.0
+--luaU.FORMAT_VERSION = 0         -- 0 is official version. yeah I know I'm a liar.
+
+-- a multiple of PI for testing native format
+-- multiplying by 1E7 gives non-trivial integer values
+--luaU.TEST_NUMBER = 3.14159265358979323846E7
+
+--[[--------------------------------------------------------------------
+-- Additional functions to handle chunk writing
+-- * to use make_setS and make_setF, see test_ldump.lua elsewhere
+----------------------------------------------------------------------]]
+
+------------------------------------------------------------------------
+-- works like the lobject.h version except that TObject used in these
+-- scripts only has a 'value' field, no 'tt' field (native types used)
+------------------------------------------------------------------------
+function luaU:ttype(o)
+  local tt = type(o.value)
+  if     tt == "number"  then return self.LUA_TNUMBER
+  elseif tt == "string"  then return self.LUA_TSTRING
+  elseif tt == "nil"     then return self.LUA_TNIL
+  elseif tt == "boolean" then return self.LUA_TBOOLEAN
+  else
+    return self.LUA_TNONE  -- the rest should not appear
+  end
+end
+
+------------------------------------------------------------------------
+-- create a chunk writer that writes to a string
+-- * returns the writer function and a table containing the string
+-- * to get the final result, look in buff.data
+------------------------------------------------------------------------
+function luaU:make_setS()
+  local buff = {}
+        buff.data = ""
+  local writer =
+    function(s, buff)  -- chunk writer
+      if not s then return end
+      buff.data = buff.data..s
+    end
+  return writer, buff
+end
+
+------------------------------------------------------------------------
+-- create a chunk writer that writes to a file
+-- * returns the writer function and a table containing the file handle
+-- * if a nil is passed, then writer should close the open file
+------------------------------------------------------------------------
+function luaU:make_setF(filename)
+  local buff = {}
+        buff.h = io.open(filename, "wb")
+  if not buff.h then return nil end
+  local writer =
+    function(s, buff)  -- chunk writer
+      if not buff.h then return end
+      if not s then buff.h:close(); return end
+      buff.h:write(s)
+    end
+  return writer, buff
+end
+
+-----------------------------------------------------------------------
+-- converts a IEEE754 double number to an 8-byte little-endian string
+-- * luaU:from_double() and luaU:from_int() are from ChunkBake project
+-- * supports +/- Infinity, but not denormals or NaNs
+-----------------------------------------------------------------------
+function luaU:from_double(x)
+  local function grab_byte(v)
+    return math.floor(v / 256),
+           string.char(math.mod(math.floor(v), 256))
+  end
+  local sign = 0
+  if x < 0 then sign = 1; x = -x end
+  local mantissa, exponent = math.frexp(x)
+  if x == 0 then -- zero
+    mantissa, exponent = 0, 0
+  elseif x == 1/0 then
+    mantissa, exponent = 0, 2047
+  else
+    mantissa = (mantissa * 2 - 1) * math.ldexp(0.5, 53)
+    exponent = exponent + 1022
+  end
+  local v, byte = "" -- convert to bytes
+  x = mantissa
+  for i = 1,6 do
+    x, byte = grab_byte(x); v = v..byte -- 47:0
+  end
+  x, byte = grab_byte(exponent * 16 + x); v = v..byte -- 55:48
+  x, byte = grab_byte(sign * 128 + x); v = v..byte -- 63:56
+  return v
+end
+
+-----------------------------------------------------------------------
+-- converts a number to a little-endian 32-bit integer string
+-- * input value assumed to not overflow, can be signed/unsigned
+-----------------------------------------------------------------------
+function luaU:from_int(x, size)
+  local v = ""
+  x = math.floor(x)
+  if x >= 0 then
+    for i = 1, size do
+      v = v..string.char(math.mod(x, 256)); x = math.floor(x / 256)
+    end
+  else -- x < 0
+    x = -x
+    local carry = 1
+    for i = 1, size do
+      local c = 255 - math.mod(x, 256) + carry
+      if c == 256 then c = 0; carry = 1 else carry = 0 end
+      v = v..string.char(c); x = math.floor(x / 256)
+    end
+  end
+  return v
+end
+
+--[[--------------------------------------------------------------------
+-- Functions to make a binary chunk
+-- * many functions have the size parameter removed, since output is
+--   in the form of a string and some sizes are implicit or hard-coded
+-- * luaU:DumpVector has been deleted (used in DumpCode & DumpLines)
+----------------------------------------------------------------------]]
+
+------------------------------------------------------------------------
+-- dump a block of literal bytes
+------------------------------------------------------------------------
+function luaU:DumpLiteral(s, D) self:DumpBlock(s, D) end
+
+--[[--------------------------------------------------------------------
+-- struct DumpState:
+--   L  -- lua_State (not used in this script)
+--   write  -- lua_Chunkwriter (chunk writer function)
+--   data  -- void* (chunk writer context or data already written)
+----------------------------------------------------------------------]]
+
+------------------------------------------------------------------------
+-- dumps a block of bytes
+-- * lua_unlock(D.L), lua_lock(D.L) deleted
+------------------------------------------------------------------------
+function luaU:DumpBlock(b, D) D.write(b, D.data) end
+
+------------------------------------------------------------------------
+-- dumps a single byte
+------------------------------------------------------------------------
+function luaU:DumpByte(y, D)
+  self:DumpBlock(string.char(y), D)
+end
+
+------------------------------------------------------------------------
+-- dumps a 32-bit signed integer (for int)
+------------------------------------------------------------------------
+function luaU:DumpInt(x, D)
+  self:DumpBlock(self:from_int(x, format.int_size), D)
+end
+
+------------------------------------------------------------------------
+-- dumps a 32-bit unsigned integer (for size_t)
+------------------------------------------------------------------------
+function luaU:DumpSize(x, D)
+  self:DumpBlock(self:from_int(x, format.size_t_size), D)
+end
+
+------------------------------------------------------------------------
+-- dumps a LUA_NUMBER (hard-coded as a double)
+------------------------------------------------------------------------
+function luaU:DumpNumber(x, D)
+   if format.integral then
+      self:DumpBlock(self:from_int(x, format.number_size), D)
+   else
+      self:DumpBlock(self:from_double(x), D)
+   end
+end
+
+------------------------------------------------------------------------
+-- dumps a Lua string
+------------------------------------------------------------------------
+function luaU:DumpString(s, D)
+  if s == nil then
+    self:DumpSize(0, D)
+  else
+    s = s.."\0"  -- include trailing '\0'
+    self:DumpSize(string.len(s), D)
+    self:DumpBlock(s, D)
+  end
+end
+
+------------------------------------------------------------------------
+-- dumps instruction block from function prototype
+------------------------------------------------------------------------
+function luaU:DumpCode(f, D)
+  local n = f.sizecode
+  self:DumpInt(n, D)
+  --was DumpVector
+  for i = 0, n - 1 do
+    self:DumpBlock(luaP:Instruction(f.code[i]), D)
+  end
+end
+
+------------------------------------------------------------------------
+-- dumps local variable names from function prototype
+------------------------------------------------------------------------
+function luaU:DumpLocals(f, D)
+  local n = f.sizelocvars
+  self:DumpInt(n, D)
+  for i = 0, n - 1 do
+    -- Dirty temporary fix: 
+    -- `Stat{ } keeps properly count of the number of local vars,
+    -- but fails to keep score of their debug info (names).
+    -- It therefore might happen that #f.localvars < f.sizelocvars, or
+    -- that a variable's startpc and endpc fields are left unset.
+    -- FIXME: This might not be needed anymore, check the bug report
+    --        by J. Belmonte.
+    local var = f.locvars[i]
+    if not var then break end 
+    -- printf("[DUMPLOCALS] dumping local var #%i = %s", i, table.tostring(var))
+    self:DumpString(var.varname, D)
+    self:DumpInt(var.startpc or 0, D)
+    self:DumpInt(var.endpc or 0, D)
+  end
+end
+
+------------------------------------------------------------------------
+-- dumps line information from function prototype
+------------------------------------------------------------------------
+function luaU:DumpLines(f, D)
+  local n = f.sizelineinfo
+  self:DumpInt(n, D)
+  --was DumpVector
+  for i = 0, n - 1 do
+    self:DumpInt(f.lineinfo[i], D)  -- was DumpBlock
+    --print(i, f.lineinfo[i])
+  end
+end
+
+------------------------------------------------------------------------
+-- dump upvalue names from function prototype
+------------------------------------------------------------------------
+function luaU:DumpUpvalues(f, D)
+  local n = f.sizeupvalues
+  self:DumpInt(n, D)
+  for i = 0, n - 1 do
+    self:DumpString(f.upvalues[i], D)
+  end
+end
+
+------------------------------------------------------------------------
+-- dump constant pool from function prototype
+-- * nvalue(o) and tsvalue(o) macros removed
+------------------------------------------------------------------------
+function luaU:DumpConstants(f, D)
+  local n = f.sizek
+  self:DumpInt(n, D)
+  for i = 0, n - 1 do
+    local o = f.k[i]  -- TObject
+    local tt = self:ttype(o)
+    assert (tt >= 0)
+    self:DumpByte(tt, D)
+    if tt == self.LUA_TNUMBER then
+       self:DumpNumber(o.value, D)
+    elseif tt == self.LUA_TSTRING then
+       self:DumpString(o.value, D)
+    elseif tt == self.LUA_TBOOLEAN then
+       self:DumpByte (o.value and 1 or 0, D)
+    elseif tt == self.LUA_TNIL then
+    else
+      assert(false)  -- cannot happen
+    end
+  end
+end
+
+
+function luaU:DumpProtos (f, D)
+  local n = f.sizep
+  assert (n)
+  self:DumpInt(n, D)
+  for i = 0, n - 1 do
+    self:DumpFunction(f.p[i], f.source, D)
+  end
+end
+
+function luaU:DumpDebug(f, D)
+  self:DumpLines(f, D)
+  self:DumpLocals(f, D)
+  self:DumpUpvalues(f, D)
+end
+
+
+------------------------------------------------------------------------
+-- dump child function prototypes from function prototype
+--FF completely reworked for 5.1 format
+------------------------------------------------------------------------
+function luaU:DumpFunction(f, p, D)
+   -- print "Dumping function:"
+   -- table.print(f, 60)
+
+  local source = f.source
+  if source == p then source = nil end
+  self:DumpString(source, D)
+  self:DumpInt(f.lineDefined, D)
+  self:DumpInt(f.lastLineDefined or 42, D)
+  self:DumpByte(f.nups, D)
+  self:DumpByte(f.numparams, D)
+  self:DumpByte(f.is_vararg, D)
+  self:DumpByte(f.maxstacksize, D)
+  self:DumpCode(f, D)
+  self:DumpConstants(f, D)
+  self:DumpProtos( f, D)
+  self:DumpDebug(f, D)
+end
+
+------------------------------------------------------------------------
+-- dump Lua header section (some sizes hard-coded)
+--FF: updated for version 5.1
+------------------------------------------------------------------------
+function luaU:DumpHeader(D)
+  self:DumpLiteral(format.header, D)
+end
+
+------------------------------------------------------------------------
+-- dump function as precompiled chunk
+-- * w, data are created from make_setS, make_setF
+--FF: suppressed extraneous [L] param
+------------------------------------------------------------------------
+function luaU:dump (Main, w, data)
+  local D = {}  -- DumpState
+  D.write = w
+  D.data = data
+  self:DumpHeader(D)
+  self:DumpFunction(Main, nil, D)
+  -- added: for a chunk writer writing to a file, this final call with
+  -- nil data is to indicate to the writer to close the file
+  D.write(nil, D.data)
+end
+
+------------------------------------------------------------------------
+-- find byte order (from lundump.c)
+-- * hard-coded to little-endian
+------------------------------------------------------------------------
+function luaU:endianness()
+  return 1
+end
+
+-- FIXME: ugly concat-base generation in [make_setS], bufferize properly! 
+function dump_string (proto)
+   local writer, buff = luaU:make_setS()
+   luaU:dump (proto, writer, buff)
+   return buff.data
+end
+
+-- FIXME: [make_setS] sucks, perform synchronous file writing
+-- Now unused
+function dump_file (proto, filename)
+   local writer, buff = luaU:make_setS()
+   luaU:dump (proto, writer, buff)
+   local file = io.open (filename, "wb")
+   file:write (buff.data)
+   io.close(file)
+   if UNIX_SHARPBANG then os.execute ("chmod a+x "..filename) end
+end

lualibs/metalua/lexer.lua

@@ -0,0 +1,511 @@
+----------------------------------------------------------------------
+-- Metalua:  $Id: mll.lua,v 1.3 2006/11/15 09:07:50 fab13n Exp $
+--
+-- Summary: generic Lua-style lexer definition. You need this plus
+-- some keyword additions to create the complete Lua lexer,
+-- as is done in mlp_lexer.lua.
+--
+-- TODO: 
+--
+-- * Make it easy to define new flavors of strings. Replacing the
+--   lexer.patterns.long_string regexp by an extensible list, with
+--   customizable token tag, would probably be enough. Maybe add:
+--   + an index of capture for the regexp, that would specify 
+--     which capture holds the content of the string-like token
+--   + a token tag
+--   + or a string->string transformer function.
+--
+-- * There are some _G.table to prevent a namespace clash which has
+--   now disappered. remove them.
+----------------------------------------------------------------------
+--
+-- Copyright (c) 2006, Fabien Fleutot <metalua@gmail.com>.
+--
+-- This software is released under the MIT Licence, see licence.txt
+-- for details.
+--
+----------------------------------------------------------------------
+
+module ("lexer", package.seeall)
+
+-- don't load metalua.runtime as it loads metalua.base, which pollutes
+-- global namespace and overwrites pairs/ipairs -- PK 6/4/2012
+require 'metalua.table2'
+
+lexer = { alpha={ }, sym={ } }
+lexer.__index=lexer
+
+local debugf = function() end
+--local debugf=printf
+
+----------------------------------------------------------------------
+-- Patterns used by [lexer:extract] to decompose the raw string into
+-- correctly tagged tokens.
+----------------------------------------------------------------------
+lexer.patterns = {
+   spaces              = "^[ \r\n\t]*()",
+   short_comment       = "^%-%-([^\n]*)()\n",
+   final_short_comment = "^%-%-([^\n]*)()$",
+   long_comment        = "^%-%-%[(=*)%[\n?(.-)%]%1%]()",
+   long_string         = "^%[(=*)%[\n?(.-)%]%1%]()",
+   number_mantissa     = { "^%d+%.?%d*()", "^%d*%.%d+()" },
+   number_exponant     = "^[eE][%+%-]?%d+()",
+   number_hex          = "^0[xX]%x+()",
+   word                = "^([%a_][%w_]*)()"
+}
+
+----------------------------------------------------------------------
+-- unescape a whole string, applying [unesc_digits] and
+-- [unesc_letter] as many times as required.
+----------------------------------------------------------------------
+local function unescape_string (s)
+
+   -- Turn the digits of an escape sequence into the corresponding
+   -- character, e.g. [unesc_digits("123") == string.char(123)].
+   local function unesc_digits (backslashes, digits)
+      if #backslashes%2==0 then
+         -- Even number of backslashes, they escape each other, not the digits.
+         -- Return them so that unesc_letter() can treaat them
+         return backslashes..digits
+      else
+         -- Remove the odd backslash, which escapes the number sequence.
+         -- The rest will be returned and parsed by unesc_letter()
+         backslashes = backslashes :sub (1,-2)
+      end
+      local k, j, i = digits:reverse():byte(1, 3)
+      local z = _G.string.byte "0"
+      local code = (k or z) + 10*(j or z) + 100*(i or z) - 111*z
+      if code > 255 then 
+      	 error ("Illegal escape sequence '\\"..digits..
+                "' in string: ASCII codes must be in [0..255]") 
+      end
+      return backslashes .. string.char (code)
+   end
+
+   -- Take a letter [x], and returns the character represented by the 
+   -- sequence ['\\'..x], e.g. [unesc_letter "n" == "\n"].
+   local function unesc_letter(x)
+      local t = { 
+         a = "\a", b = "\b", f = "\f",
+         n = "\n", r = "\r", t = "\t", v = "\v",
+         ["\\"] = "\\", ["'"] = "'", ['"'] = '"', ["\n"] = "\n" }
+      return t[x] or error([[Unknown escape sequence '\]]..x..[[']])
+   end
+
+   return s
+      :gsub ("(\\+)([0-9][0-9]?[0-9]?)", unesc_digits)
+      :gsub ("\\(%D)",unesc_letter)
+end
+
+lexer.extractors = {
+   "skip_whitespaces_and_comments",
+   "extract_short_string", "extract_word", "extract_number", 
+   "extract_long_string", "extract_symbol" }
+
+lexer.token_metatable = { 
+--         __tostring = function(a) 
+--            return string.format ("`%s{'%s'}",a.tag, a[1]) 
+--         end 
+} 
+      
+lexer.lineinfo_metatable = { }
+
+----------------------------------------------------------------------
+-- Really extract next token fron the raw string 
+-- (and update the index).
+-- loc: offset of the position just after spaces and comments
+-- previous_i: offset in src before extraction began
+----------------------------------------------------------------------
+function lexer:extract ()
+   local previous_i = self.i
+   local loc = self.i
+   local eof, token
+
+   -- Put line info, comments and metatable around the tag and content
+   -- provided by extractors, thus returning a complete lexer token.
+   -- first_line: line # at the beginning of token
+   -- first_column_offset: char # of the last '\n' before beginning of token
+   -- i: scans from beginning of prefix spaces/comments to end of token.
+   local function build_token (tag, content)
+      assert (tag and content)
+      local i, first_line, first_column_offset, previous_line_length =
+         previous_i, self.line, self.column_offset, nil
+
+      -- update self.line and first_line. i := indexes of '\n' chars
+      while true do
+         i = self.src :find ("\n", i+1, true)
+         if not i or i>self.i then break end -- no more '\n' until end of token
+         previous_line_length = i - self.column_offset
+         if loc and i <= loc then -- '\n' before beginning of token
+            first_column_offset = i
+            first_line = first_line+1 
+         end
+         self.line   = self.line+1 
+         self.column_offset = i 
+      end
+
+      -- lineinfo entries: [1]=line, [2]=column, [3]=char, [4]=filename
+      local fli = { first_line, loc-first_column_offset, loc, self.src_name }
+      local lli = { self.line, self.i-self.column_offset-1, self.i-1, self.src_name }
+      --Pluto barfes when the metatable is set:(
+      setmetatable(fli, lexer.lineinfo_metatable)
+      setmetatable(lli, lexer.lineinfo_metatable)
+      local a = { tag = tag, lineinfo = { first=fli, last=lli }, content } 
+      if lli[2]==-1 then lli[1], lli[2] = lli[1]-1, previous_line_length-1 end
+      if #self.attached_comments > 0 then 
+         a.lineinfo.comments = self.attached_comments 
+         fli.comments = self.attached_comments
+         if self.lineinfo_last then
+            self.lineinfo_last.comments = self.attached_comments
+         end
+      end
+      self.attached_comments = { }
+      return setmetatable (a, self.token_metatable)
+   end --</function build_token>
+
+   for ext_idx, extractor in ipairs(self.extractors) do
+      -- printf("method = %s", method)
+      local tag, content = self [extractor] (self)
+      -- [loc] is placed just after the leading whitespaces and comments;
+      -- for this to work, the whitespace extractor *must be* at index 1.
+      if ext_idx==1 then loc = self.i end
+
+      if tag then 
+         --printf("`%s{ %q }\t%i", tag, content, loc);
+         return build_token (tag, content) 
+      end
+   end
+
+   error "None of the lexer extractors returned anything!"
+end   
+
+----------------------------------------------------------------------
+-- skip whites and comments
+-- FIXME: doesn't take into account:
+-- - unterminated long comments
+-- - short comments at last line without a final \n
+----------------------------------------------------------------------
+function lexer:skip_whitespaces_and_comments()
+   local table_insert = _G.table.insert
+   repeat -- loop as long as a space or comment chunk is found
+      local _, j
+      local again = false
+      local last_comment_content = nil
+      -- skip spaces
+      self.i = self.src:match (self.patterns.spaces, self.i)
+      -- skip a long comment if any
+      _, last_comment_content, j = 
+         self.src :match (self.patterns.long_comment, self.i)
+      if j then 
+         table_insert(self.attached_comments, 
+                         {last_comment_content, self.i, j, "long"})
+         self.i=j; again=true 
+      end
+      -- skip a short comment if any
+      last_comment_content, j = self.src:match (self.patterns.short_comment, self.i)
+      if j then
+         table_insert(self.attached_comments, 
+                         {last_comment_content, self.i, j, "short"})
+         self.i=j; again=true 
+      end
+      if self.i>#self.src then return "Eof", "eof" end
+   until not again
+
+   if self.src:match (self.patterns.final_short_comment, self.i) then 
+      return "Eof", "eof" end
+   --assert (not self.src:match(self.patterns.short_comment, self.i))
+   --assert (not self.src:match(self.patterns.long_comment, self.i))
+   -- --assert (not self.src:match(self.patterns.spaces, self.i))
+   return
+end
+
+----------------------------------------------------------------------
+-- extract a '...' or "..." short string
+----------------------------------------------------------------------
+function lexer:extract_short_string()
+   -- [k] is the first unread char, [self.i] points to [k] in [self.src]
+   local j, k = self.i, self.src :sub (self.i,self.i)
+   if k~="'" and k~='"' then return end
+   local i = self.i + 1
+   local j = i
+   while true do
+      -- k = opening char: either simple-quote or double-quote
+      -- i = index of beginning-of-string
+      -- x = next "interesting" character
+      -- j = position after interesting char
+      -- y = char just after x
+      local x, y
+      x, j, y = self.src :match ("([\\\r\n"..k.."])()(.?)", j)
+      if x == '\\' then j=j+1  -- don't parse escaped char
+      elseif x == k then break -- unescaped end of string
+      else -- eof or '\r' or '\n' reached before end of string
+         assert (not x or x=="\r" or x=="\n")
+         error "Unterminated string"
+      end
+   end
+   self.i = j
+
+   return "String", unescape_string (self.src:sub (i,j-2))
+end
+
+----------------------------------------------------------------------
+--
+----------------------------------------------------------------------
+function lexer:extract_word()
+   -- Id / keyword
+   local word, j = self.src:match (self.patterns.word, self.i)
+   if word then
+      self.i = j
+      if self.alpha [word] then return "Keyword", word
+      else return "Id", word end
+   end
+end
+
+----------------------------------------------------------------------
+--
+----------------------------------------------------------------------
+function lexer:extract_number()
+   -- Number
+   local j = self.src:match(self.patterns.number_hex, self.i)
+   if not j then
+      j = self.src:match (self.patterns.number_mantissa[1], self.i) or
+          self.src:match (self.patterns.number_mantissa[2], self.i)
+      if j then
+         j = self.src:match (self.patterns.number_exponant, j) or j;
+      end
+   end
+   if not j then return end
+   -- Number found, interpret with tonumber() and return it
+   local n = tonumber (self.src:sub (self.i, j-1))
+   self.i = j
+   return "Number", n
+end
+
+----------------------------------------------------------------------
+--
+----------------------------------------------------------------------
+function lexer:extract_long_string()
+   -- Long string
+   local _, content, j = self.src:match (self.patterns.long_string, self.i)
+   if j then self.i = j; return "String", content end
+end
+
+----------------------------------------------------------------------
+--
+----------------------------------------------------------------------
+function lexer:extract_symbol()
+   -- compound symbol
+   local k = self.src:sub (self.i,self.i)
+   local symk = self.sym [k]
+   if not symk then 
+      self.i = self.i + 1
+      return "Keyword", k
+   end
+   for _, sym in pairs (symk) do
+      if sym == self.src:sub (self.i, self.i + #sym - 1) then 
+         self.i = self.i + #sym; 
+         return "Keyword", sym
+      end
+   end
+   -- single char symbol
+   self.i = self.i+1
+   return "Keyword", k
+end
+
+----------------------------------------------------------------------
+-- Add a keyword to the list of keywords recognized by the lexer.
+----------------------------------------------------------------------
+function lexer:add (w, ...)
+   assert(not ..., "lexer:add() takes only one arg, although possibly a table")
+   if type (w) == "table" then
+      for _, x in ipairs (w) do self:add (x) end
+   else
+      if w:match (self.patterns.word .. "$") then self.alpha [w] = true
+      elseif w:match "^%p%p+$" then 
+         local k = w:sub(1,1)
+         local list = self.sym [k]
+         if not list then list = { }; self.sym [k] = list end
+         _G.table.insert (list, w)
+      elseif w:match "^%p$" then return
+      else error "Invalid keyword" end
+   end
+end
+
+----------------------------------------------------------------------
+-- Return the [n]th next token, without consumming it.
+-- [n] defaults to 1. If it goes pass the end of the stream, an EOF
+-- token is returned.
+----------------------------------------------------------------------
+function lexer:peek (n)
+   if not n then n=1 end
+   if n > #self.peeked then
+      for i = #self.peeked+1, n do
+         self.peeked [i] = self:extract()
+      end
+   end
+  return self.peeked [n]
+end
+
+----------------------------------------------------------------------
+-- Return the [n]th next token, removing it as well as the 0..n-1
+-- previous tokens. [n] defaults to 1. If it goes pass the end of the
+-- stream, an EOF token is returned.
+----------------------------------------------------------------------
+function lexer:next (n)
+   n = n or 1
+   self:peek (n)
+   local a
+   for i=1,n do 
+      a = _G.table.remove (self.peeked, 1) 
+      if a then 
+         --debugf ("lexer:next() ==> %s %s",
+         --        table.tostring(a), tostring(a))
+      end
+      self.lastline = a.lineinfo.last[1]
+   end
+   self.lineinfo_last = a.lineinfo.last
+   return a or eof_token
+end
+
+----------------------------------------------------------------------
+-- Returns an object which saves the stream's current state.
+----------------------------------------------------------------------
+-- FIXME there are more fields than that to save
+function lexer:save () return { self.i; _G.table.cat(self.peeked) } end
+
+----------------------------------------------------------------------
+-- Restore the stream's state, as saved by method [save].
+----------------------------------------------------------------------
+-- FIXME there are more fields than that to restore
+function lexer:restore (s) self.i=s[1]; self.peeked=s[2] end
+
+----------------------------------------------------------------------
+-- Resynchronize: cancel any token in self.peeked, by emptying the
+-- list and resetting the indexes
+----------------------------------------------------------------------
+function lexer:sync()
+   local p1 = self.peeked[1]
+   if p1 then 
+      li = p1.lineinfo.first
+      self.line, self.i = li[1], li[3]
+      self.column_offset = self.i - li[2]
+      self.peeked = { }
+      self.attached_comments = p1.lineinfo.first.comments or { }
+   end
+end
+
+----------------------------------------------------------------------
+-- Take the source and offset of an old lexer.
+----------------------------------------------------------------------
+function lexer:takeover(old)
+   self:sync()
+   self.line, self.column_offset, self.i, self.src, self.attached_comments =
+      old.line, old.column_offset, old.i, old.src, old.attached_comments
+   return self
+end
+
+-- function lexer:lineinfo()
+-- 	if self.peeked[1] then return self.peeked[1].lineinfo.first
+--     else return { self.line, self.i-self.column_offset, self.i } end
+-- end
+
+
+----------------------------------------------------------------------
+-- Return the current position in the sources. This position is between
+-- two tokens, and can be within a space / comment area, and therefore
+-- have a non-null width. :lineinfo_left() returns the beginning of the
+-- separation area, :lineinfo_right() returns the end of that area.
+--
+--    ____ last consummed token    ____ first unconsummed token
+--   /                            /
+-- XXXXX  <spaces and comments> YYYYY
+--      \____                    \____
+--           :lineinfo_left()         :lineinfo_right()
+----------------------------------------------------------------------
+function lexer:lineinfo_right()
+   return self:peek(1).lineinfo.first
+end
+
+function lexer:lineinfo_left()
+   return self.lineinfo_last
+end
+
+----------------------------------------------------------------------
+-- Create a new lexstream.
+----------------------------------------------------------------------
+function lexer:newstream (src_or_stream, name)
+   name = name or "?"
+   if type(src_or_stream)=='table' then -- it's a stream
+      return setmetatable ({ }, self) :takeover (src_or_stream)
+   elseif type(src_or_stream)=='string' then -- it's a source string
+      local src = src_or_stream
+      local stream = { 
+         src_name      = name;   -- Name of the file
+         src           = src;    -- The source, as a single string
+         peeked        = { };    -- Already peeked, but not discarded yet, tokens
+         i             = 1;      -- Character offset in src
+         line          = 1;      -- Current line number
+         column_offset = 0;      -- distance from beginning of file to last '\n'
+         attached_comments = { },-- comments accumulator
+         lineinfo_last = { 1, 1, 1, name }
+      }
+      setmetatable (stream, self)
+
+      -- skip initial sharp-bang for unix scripts
+      -- FIXME: redundant with mlp.chunk()
+      if src and src :match "^#" then stream.i = src :find "\n" + 1 end
+      return stream
+   else
+      assert(false, ":newstream() takes a source string or a stream, not a "..
+                    type(src_or_stream))
+   end
+end
+
+----------------------------------------------------------------------
+-- if there's no ... args, return the token a (whose truth value is
+-- true) if it's a `Keyword{ }, or nil.  If there are ... args, they
+-- have to be strings. if the token a is a keyword, and it's content
+-- is one of the ... args, then returns it (it's truth value is
+-- true). If no a keyword or not in ..., return nil.
+----------------------------------------------------------------------
+function lexer:is_keyword (a, ...)
+   if not a or a.tag ~= "Keyword" then return false end
+   local words = {...}
+   if #words == 0 then return a[1] end
+   for _, w in ipairs (words) do
+      if w == a[1] then return w end
+   end
+   return false
+end
+
+----------------------------------------------------------------------
+-- Cause an error if the next token isn't a keyword whose content
+-- is listed among ... args (which have to be strings).
+----------------------------------------------------------------------
+function lexer:check (...)
+   local words = {...}
+   local a = self:next()
+   local function err ()
+      error ("Got " .. tostring (a) .. 
+             ", expected one of these keywords : '" ..
+             _G.table.concat (words,"', '") .. "'") end
+          
+   if not a or a.tag ~= "Keyword" then err () end
+   if #words == 0 then return a[1] end
+   for _, w in ipairs (words) do
+       if w == a[1] then return w end
+   end
+   err ()
+end
+
+----------------------------------------------------------------------
+-- 
+----------------------------------------------------------------------
+function lexer:clone()
+   local clone = {
+      alpha = table.deep_copy(self.alpha),
+      sym   = table.deep_copy(self.sym) }
+   setmetatable(clone, self)
+   clone.__index = clone
+   return clone
+end

lualibs/metalua/lopcodes.lua

@@ -0,0 +1,440 @@
+----------------------------------------------------------------------
+--
+-- WARNING! You're entering a hackish area, proceed at your own risks!
+--
+-- This code results from the borrowing, then ruthless abuse, of
+-- Yueliang's implementation of Lua 5.0 compiler. I claim
+-- responsibility for all of the ugly, dirty stuff that you might spot
+-- in it.
+--
+-- Eventually, this code will be rewritten, either in Lua or more
+-- probably in C. Meanwhile, if you're interested into digging
+-- metalua's sources, this is not the best part to invest your time
+-- on.
+--
+-- End of warning.
+--
+----------------------------------------------------------------------
+
+--[[--------------------------------------------------------------------
+
+  $Id$
+
+  lopcodes.lua
+  Lua 5 virtual machine opcodes in Lua
+  This file is part of Yueliang.
+
+  Copyright (c) 2005 Kein-Hong Man <khman@users.sf.net>
+  The COPYRIGHT file describes the conditions
+  under which this software may be distributed.
+
+  See the ChangeLog for more information.
+
+------------------------------------------------------------------------
+
+  [FF] Slightly modified, mainly to produce Lua 5.1 bytecode.
+
+----------------------------------------------------------------------]]
+
+--[[--------------------------------------------------------------------
+-- Notes:
+-- * an Instruction is a table with OP, A, B, C, Bx elements; this
+--   should allow instruction handling to work with doubles and ints
+-- * Added:
+--   luaP:Instruction(i): convert field elements to a 4-char string
+--   luaP:DecodeInst(x): convert 4-char string into field elements
+-- * WARNING luaP:Instruction outputs instructions encoded in little-
+--   endian form and field size and positions are hard-coded
+----------------------------------------------------------------------]]
+
+module("bytecode", package.seeall)
+
+local function debugf() end
+
+luaP = { }
+
+--[[
+===========================================================================
+  We assume that instructions are unsigned numbers.
+  All instructions have an opcode in the first 6 bits.
+  Instructions can have the following fields:
+        'A' : 8 bits
+        'B' : 9 bits
+        'C' : 9 bits
+        'Bx' : 18 bits ('B' and 'C' together)
+        'sBx' : signed Bx
+
+  A signed argument is represented in excess K; that is, the number
+  value is the unsigned value minus K. K is exactly the maximum value
+  for that argument (so that -max is represented by 0, and +max is
+  represented by 2*max), which is half the maximum for the corresponding
+  unsigned argument.
+===========================================================================
+--]]
+
+luaP.OpMode = {"iABC", "iABx", "iAsBx"}  -- basic instruction format
+
+------------------------------------------------------------------------
+-- size and position of opcode arguments.
+-- * WARNING size and position is hard-coded elsewhere in this script
+------------------------------------------------------------------------
+luaP.SIZE_C  = 9
+luaP.SIZE_B  = 9
+luaP.SIZE_Bx = luaP.SIZE_C + luaP.SIZE_B
+luaP.SIZE_A  = 8
+
+luaP.SIZE_OP = 6
+
+luaP.POS_C  = luaP.SIZE_OP
+luaP.POS_B  = luaP.POS_C + luaP.SIZE_C
+luaP.POS_Bx = luaP.POS_C
+luaP.POS_A  = luaP.POS_B + luaP.SIZE_B
+
+--FF from 5.1
+luaP.BITRK = 2^(luaP.SIZE_B - 1)
+function luaP:ISK(x) return x >= self.BITRK end
+luaP.MAXINDEXRK = luaP.BITRK - 1
+function luaP:RKASK(x)
+   if x < self.BITRK then return x+self.BITRK else return x end
+end
+
+
+
+------------------------------------------------------------------------
+-- limits for opcode arguments.
+-- we use (signed) int to manipulate most arguments,
+-- so they must fit in BITS_INT-1 bits (-1 for sign)
+------------------------------------------------------------------------
+-- removed "#if SIZE_Bx < BITS_INT-1" test, assume this script is
+-- running on a Lua VM with double or int as LUA_NUMBER
+
+luaP.MAXARG_Bx  = math.ldexp(1, luaP.SIZE_Bx) - 1
+luaP.MAXARG_sBx = math.floor(luaP.MAXARG_Bx / 2)  -- 'sBx' is signed
+
+luaP.MAXARG_A = math.ldexp(1, luaP.SIZE_A) - 1
+luaP.MAXARG_B = math.ldexp(1, luaP.SIZE_B) - 1
+luaP.MAXARG_C = math.ldexp(1, luaP.SIZE_C) - 1
+
+-- creates a mask with 'n' 1 bits at position 'p'
+-- MASK1(n,p) deleted
+-- creates a mask with 'n' 0 bits at position 'p'
+-- MASK0(n,p) deleted
+
+--[[--------------------------------------------------------------------
+  Visual representation for reference:
+
+   31     |     |     |          0      bit position
+    +-----+-----+-----+----------+
+    |  B  |  C  |  A  |  Opcode  |      iABC format
+    +-----+-----+-----+----------+
+    -  9  -  9  -  8  -    6     -      field sizes
+    +-----+-----+-----+----------+
+    |   [s]Bx   |  A  |  Opcode  |      iABx | iAsBx format
+    +-----+-----+-----+----------+
+----------------------------------------------------------------------]]
+
+------------------------------------------------------------------------
+-- the following macros help to manipulate instructions
+-- * changed to a table object representation, very clean compared to
+--   the [nightmare] alternatives of using a number or a string
+------------------------------------------------------------------------
+
+-- these accept or return opcodes in the form of string names
+function luaP:GET_OPCODE(i) return self.ROpCode[i.OP] end
+function luaP:SET_OPCODE(i, o) i.OP = self.OpCode[o] end
+
+function luaP:GETARG_A(i) return i.A end
+function luaP:SETARG_A(i, u) i.A = u end
+
+function luaP:GETARG_B(i) return i.B end
+function luaP:SETARG_B(i, b) i.B = b end
+
+function luaP:GETARG_C(i) return i.C end
+function luaP:SETARG_C(i, b) i.C = b end
+
+function luaP:GETARG_Bx(i) return i.Bx end
+function luaP:SETARG_Bx(i, b) i.Bx = b end
+
+function luaP:GETARG_sBx(i) return i.Bx - self.MAXARG_sBx end
+function luaP:SETARG_sBx(i, b) i.Bx = b + self.MAXARG_sBx end
+
+function luaP:CREATE_ABC(o,a,b,c)
+  return {OP = self.OpCode[o], A = a, B = b, C = c}
+end
+
+function luaP:CREATE_ABx(o,a,bc)
+  return {OP = self.OpCode[o], A = a, Bx = bc}
+end
+
+------------------------------------------------------------------------
+-- Bit shuffling stuffs
+------------------------------------------------------------------------
+
+if false and pcall (require, 'bit') then
+   ------------------------------------------------------------------------
+   -- Return a 4-char string little-endian encoded form of an instruction
+   ------------------------------------------------------------------------
+   function luaP:Instruction(i)
+      --FIXME
+   end
+else
+   ------------------------------------------------------------------------   
+   -- Version without bit manipulation library.
+   ------------------------------------------------------------------------
+   local p2 = {1,2,4,8,16,32,64,128,256, 512, 1024, 2048, 4096}
+   -- keeps [n] bits from [x]
+   local function keep (x, n) return x % p2[n+1] end
+   -- shifts bits of [x] [n] places to the right
+   local function srb (x,n) return math.floor (x / p2[n+1]) end
+   -- shifts bits of [x] [n] places to the left
+   local function slb (x,n) return x * p2[n+1] end
+
+   ------------------------------------------------------------------------
+   -- Return a 4-char string little-endian encoded form of an instruction
+   ------------------------------------------------------------------------
+   function luaP:Instruction(i)
+      -- printf("Instr->string: %s %s", self.opnames[i.OP], table.tostring(i))
+      local c0, c1, c2, c3
+      -- change to OP/A/B/C format if needed
+      if i.Bx then i.C = keep (i.Bx, 9); i.B = srb (i.Bx, 9) end
+      -- c0 = 6B from opcode + 2LSB from A (flushed to MSB)
+      c0 = i.OP + slb (keep (i.A, 2), 6) 
+      -- c1 = 6MSB from A + 2LSB from C (flushed to MSB)
+      c1 = srb (i.A, 2) + slb (keep (i.C, 2), 6)
+      -- c2 = 7MSB from C + 1LSB from B (flushed to MSB)
+      c2 = srb (i.C, 2) + slb (keep (i.B, 1), 7)
+      -- c3 = 8MSB from B
+      c3 = srb (i.B, 1)
+      --printf ("Instruction:   %s %s", self.opnames[i.OP], tostringv (i))
+      --printf ("Bin encoding:  %x %x %x %x", c0, c1, c2, c3)  
+      return string.char(c0, c1, c2, c3)
+   end
+end
+------------------------------------------------------------------------
+-- decodes a 4-char little-endian string into an instruction struct
+------------------------------------------------------------------------
+function luaP:DecodeInst(x)
+  error "Not implemented"
+end
+
+------------------------------------------------------------------------
+-- invalid register that fits in 8 bits
+------------------------------------------------------------------------
+luaP.NO_REG = luaP.MAXARG_A
+
+------------------------------------------------------------------------
+-- R(x) - register
+-- Kst(x) - constant (in constant table)
+-- RK(x) == if x < MAXSTACK then R(x) else Kst(x-MAXSTACK)
+------------------------------------------------------------------------
+
+------------------------------------------------------------------------
+-- grep "ORDER OP" if you change these enums
+------------------------------------------------------------------------
+
+--[[--------------------------------------------------------------------
+Lua virtual machine opcodes (enum OpCode):
+------------------------------------------------------------------------
+name          args    description
+------------------------------------------------------------------------
+OP_MOVE       A B     R(A) := R(B)
+OP_LOADK      A Bx    R(A) := Kst(Bx)
+OP_LOADBOOL   A B C   R(A) := (Bool)B; if (C) PC++
+OP_LOADNIL    A B     R(A) := ... := R(B) := nil
+OP_GETUPVAL   A B     R(A) := UpValue[B]
+OP_GETGLOBAL  A Bx    R(A) := Gbl[Kst(Bx)]
+OP_GETTABLE   A B C   R(A) := R(B)[RK(C)]
+OP_SETGLOBAL  A Bx    Gbl[Kst(Bx)] := R(A)
+OP_SETUPVAL   A B     UpValue[B] := R(A)
+OP_SETTABLE   A B C   R(A)[RK(B)] := RK(C)
+OP_NEWTABLE   A B C   R(A) := {} (size = B,C)
+OP_SELF       A B C   R(A+1) := R(B); R(A) := R(B)[RK(C)]
+OP_ADD        A B C   R(A) := RK(B) + RK(C)
+OP_SUB        A B C   R(A) := RK(B) - RK(C)
+OP_MUL        A B C   R(A) := RK(B) * RK(C)
+OP_DIV        A B C   R(A) := RK(B) / RK(C)
+OP_POW        A B C   R(A) := RK(B) ^ RK(C)
+OP_UNM        A B     R(A) := -R(B)
+OP_NOT        A B     R(A) := not R(B)
+OP_CONCAT     A B C   R(A) := R(B).. ... ..R(C)
+OP_JMP        sBx     PC += sBx
+OP_EQ         A B C   if ((RK(B) == RK(C)) ~= A) then pc++
+OP_LT         A B C   if ((RK(B) <  RK(C)) ~= A) then pc++
+OP_LE         A B C   if ((RK(B) <= RK(C)) ~= A) then pc++
+OP_TEST       A B C   if (R(B) <=> C) then R(A) := R(B) else pc++
+OP_CALL       A B C   R(A), ... ,R(A+C-2) := R(A)(R(A+1), ... ,R(A+B-1))
+OP_TAILCALL   A B C   return R(A)(R(A+1), ... ,R(A+B-1))
+OP_RETURN     A B     return R(A), ... ,R(A+B-2)  (see note)
+OP_FORLOOP    A sBx   R(A)+=R(A+2); if R(A) <?= R(A+1) then PC+= sBx
+OP_TFORLOOP   A C     R(A+2), ... ,R(A+2+C) := R(A)(R(A+1), R(A+2));
+                      if R(A+2) ~= nil then pc++
+OP_TFORPREP   A sBx   if type(R(A)) == table then R(A+1):=R(A), R(A):=next;
+                      PC += sBx
+OP_SETLIST    A Bx    R(A)[Bx-Bx%FPF+i] := R(A+i), 1 <= i <= Bx%FPF+1
+OP_SETLISTO   A Bx    (see note)
+OP_CLOSE      A       close all variables in the stack up to (>=) R(A)
+OP_CLOSURE    A Bx    R(A) := closure(KPROTO[Bx], R(A), ... ,R(A+n))
+----------------------------------------------------------------------]]
+
+luaP.opnames = {}  -- opcode names
+luaP.OpCode = {}   -- lookup name -> number
+luaP.ROpCode = {}  -- lookup number -> name
+
+local i = 0
+for v in string.gfind([[
+MOVE -- 0
+LOADK
+LOADBOOL
+LOADNIL
+GETUPVAL
+GETGLOBAL -- 5
+GETTABLE
+SETGLOBAL
+SETUPVAL
+SETTABLE
+NEWTABLE -- 10
+SELF
+ADD
+SUB
+MUL
+DIV -- 15
+MOD
+POW
+UNM
+NOT
+LEN -- 20
+CONCAT
+JMP
+EQ
+LT
+LE -- 25
+TEST
+TESTSET
+CALL
+TAILCALL
+RETURN -- 30
+FORLOOP
+FORPREP
+TFORLOOP
+SETLIST
+CLOSE -- 35
+CLOSURE
+VARARG
+]], "[%a]+") do
+  local n = "OP_"..v
+  luaP.opnames[i] = v
+  luaP.OpCode[n] = i
+  luaP.ROpCode[i] = n
+  i = i + 1
+end
+luaP.NUM_OPCODES = i
+
+--[[
+===========================================================================
+  Notes:
+  (1) In OP_CALL, if (B == 0) then B = top. C is the number of returns - 1,
+      and can be 0: OP_CALL then sets 'top' to last_result+1, so
+      next open instruction (OP_CALL, OP_RETURN, OP_SETLIST) may use 'top'.
+
+  (2) In OP_RETURN, if (B == 0) then return up to 'top'
+
+  (3) For comparisons, B specifies what conditions the test should accept.
+
+  (4) All 'skips' (pc++) assume that next instruction is a jump
+
+  (5) OP_SETLISTO is used when the last item in a table constructor is a
+      function, so the number of elements set is up to top of stack
+===========================================================================
+--]]
+
+------------------------------------------------------------------------
+-- masks for instruction properties
+------------------------------------------------------------------------
+-- was enum OpModeMask:
+luaP.OpModeBreg = 2  -- B is a register
+luaP.OpModeBrk  = 3  -- B is a register/constant
+luaP.OpModeCrk  = 4  -- C is a register/constant
+luaP.OpModesetA = 5  -- instruction set register A
+luaP.OpModeK    = 6  -- Bx is a constant
+luaP.OpModeT    = 1  -- operator is a test
+
+------------------------------------------------------------------------
+-- get opcode mode, e.g. "iABC"
+------------------------------------------------------------------------
+function luaP:getOpMode(m)
+   --printv(m)
+   --printv(self.OpCode[m])
+   --printv(self.opmodes [self.OpCode[m]+1])
+   return self.OpMode[tonumber(string.sub(self.opmodes[self.OpCode[m] + 1], 7, 7))]
+end
+
+------------------------------------------------------------------------
+-- test an instruction property flag
+-- * b is a string, e.g. "OpModeBreg"
+------------------------------------------------------------------------
+function luaP:testOpMode(m, b)
+  return (string.sub(self.opmodes[self.OpCode[m] + 1], self[b], self[b]) == "1")
+end
+
+-- number of list items to accumulate before a SETLIST instruction
+-- (must be a power of 2)
+-- * used in lparser, lvm, ldebug, ltests
+luaP.LFIELDS_PER_FLUSH = 50 --FF updated to match 5.1
+
+-- luaP_opnames[] is set above, as the luaP.opnames table
+-- opmode(t,b,bk,ck,sa,k,m) deleted
+
+--[[--------------------------------------------------------------------
+  Legend for luaP:opmodes:
+  1 T  -> T (is a test?)
+  2 B  -> B is a register
+  3 b  -> B is an RK register/constant combination
+  4 C  -> C is an RK register/constant combination
+  5 A  -> register A is set by the opcode
+  6 K  -> Bx is a constant
+  7 m  -> 1 if iABC  layout,
+          2 if iABx  layout, 
+          3 if iAsBx layout
+----------------------------------------------------------------------]]
+
+luaP.opmodes = {
+-- TBbCAKm      opcode
+  "0100101", -- OP_MOVE      0
+  "0000112", -- OP_LOADK
+  "0000101", -- OP_LOADBOOL
+  "0100101", -- OP_LOADNIL
+  "0000101", -- OP_GETUPVAL
+  "0000112", -- OP_GETGLOBAL 5
+  "0101101", -- OP_GETTABLE
+  "0000012", -- OP_SETGLOBAL
+  "0000001", -- OP_SETUPVAL
+  "0011001", -- OP_SETTABLE
+  "0000101", -- OP_NEWTABLE 10
+  "0101101", -- OP_SELF
+  "0011101", -- OP_ADD
+  "0011101", -- OP_SUB
+  "0011101", -- OP_MUL
+  "0011101", -- OP_DIV      15
+  "0011101", -- OP_MOD
+  "0011101", -- OP_POW
+  "0100101", -- OP_UNM
+  "0100101", -- OP_NOT
+  "0100101", -- OP_LEN      20
+  "0101101", -- OP_CONCAT
+  "0000003", -- OP_JMP
+  "1011001", -- OP_EQ
+  "1011001", -- OP_LT
+  "1011001", -- OP_LE       25
+  "1000101", -- OP_TEST
+  "1100101", -- OP_TESTSET
+  "0000001", -- OP_CALL
+  "0000001", -- OP_TAILCALL
+  "0000001", -- OP_RETURN   30
+  "0000003", -- OP_FORLOOP
+  "0000103", -- OP_FORPREP
+  "1000101", -- OP_TFORLOOP
+  "0000001", -- OP_SETLIST
+  "0000001", -- OP_CLOSE    35
+  "0000102", -- OP_CLOSURE
+  "0000101"  -- OP_VARARG
+}

lualibs/metalua/metalua.lua

@@ -0,0 +1,60 @@
+-- construct proper path to load metalua modules to build
+-- an abstract syntax tree (AST)
+local file = debug.getinfo(1, "S").source
+if string.find(file, "@") == 1 then file = string.sub(file, 2) end
+package.path = package.path .. ';' .. string.gsub(file, "metalua%.lua$", "?.lua")
+
+-- these modules are sufficient to build an AST from a source file/string
+require "lexer"
+require "gg"
+require "mlp_lexer"
+require "mlp_misc"
+require "mlp_table"
+require "mlp_meta"
+require "mlp_expr"
+require "mlp_stat"
+
+-- these modules are needed to convert an AST into bytecode to execute
+require "lcode"
+require "ldump"
+require "lopcodes"
+require "compile"
+
+-- this is the compiler module that builds bytecode from an AST
+local mlc = { } 
+
+function mlc.function_of_ast (ast)
+   local  proto = bytecode.metalua_compile(ast)
+   local  dump  = bytecode.dump_string(proto)
+   local  func  = string.undump(dump) 
+   return func
+end
+
+function mlc.ast_of_luastring (src, file)
+   local  lx  = mlp.lexer:newstream(src, file or "(string)")
+   local  ast = mlp.chunk(lx)
+   return ast
+end
+
+function mlc.function_of_luastring (src, file)
+   local  ast  = mlc.ast_of_luastring(src, file)
+   local  func = mlc.function_of_ast(ast)
+   return func
+end
+
+function mlc.function_of_luafile (name)
+   local f   = io.open(name, 'r')
+   local src = f:read('*a')
+   f:close()
+   return mlc.function_of_luastring(src, "@"..name)
+end
+
+_G.mlc = mlc
+
+--[[ 
+  -- Can be used with the following code:
+  require "metalua"
+  local ast = mlc.ast_of_luastring(src)
+  local f = mlc.function_of_ast(ast)
+  f()
+]]

lualibs/metalua/metalua/base.lua

@@ -0,0 +1,104 @@
+----------------------------------------------------------------------
+----------------------------------------------------------------------
+--
+-- Base library extension
+--
+----------------------------------------------------------------------
+----------------------------------------------------------------------
+
+if not metalua then rawset(getfenv(), 'metalua', { }) end
+metalua.version             = "v-0.5"
+
+if not rawpairs then
+   rawpairs, rawipairs, rawtype = pairs, ipairs, type
+end
+
+function pairs(x)
+   assert(type(x)=='table', 'pairs() expects a table')
+   local mt = getmetatable(x)
+   if mt then
+      local mtp = mt.__pairs
+      if mtp then return mtp(x) end
+   end
+   return rawpairs(x)
+end
+
+function ipairs(x)
+   assert(type(x)=='table', 'ipairs() expects a table')
+   local mt = getmetatable(x)
+   if mt then
+      local mti = mt.__ipairs
+      if mti then return mti(x) end
+   end
+   return rawipairs(x)
+end
+
+--[[
+function type(x)
+   local mt = getmetatable(x)
+   if mt then
+      local mtt = mt.__type
+      if mtt then return mtt end
+   end
+   return rawtype(x)
+end
+]]
+
+function min (a, ...)
+   for n in values{...} do if n<a then a=n end end
+   return a
+end
+
+function max (a, ...)
+   for n in values{...} do if n>a then a=n end end
+   return a
+end
+
+function o (...)
+   local args = {...}
+   local function g (...)
+      local result = {...}
+      for i=#args, 1, -1 do result = {args[i](unpack(result))} end
+      return unpack (result)
+   end
+   return g
+end
+
+function id (...) return ... end
+function const (k) return function () return k end end
+
+function printf(...) return print(string.format(...)) end
+function eprintf(...) 
+   io.stderr:write(string.format(...).."\n") 
+end
+
+function ivalues (x)
+   assert(type(x)=='table', 'ivalues() expects a table')
+   local i = 1
+   local function iterator ()
+      local r = x[i]; i=i+1; return r
+   end
+   return iterator
+end
+
+
+function values (x)
+   assert(type(x)=='table', 'values() expects a table')
+   local function iterator (state)
+      local it
+      state.content, it = next(state.list, state.content)
+      return it
+   end
+   return iterator, { list = x }
+end
+
+function keys (x)
+   assert(type(x)=='table', 'keys() expects a table')
+   local function iterator (state)
+      local it = next(state.list, state.content)
+      state.content = it
+      return it
+   end
+   return iterator, { list = x }
+end
+

lualibs/metalua/metalua/runtime.lua

@@ -0,0 +1,3 @@
+require 'metalua.base'
+require 'metalua.table2'
+require 'metalua.string2'

lualibs/metalua/metalua/string2.lua

@@ -0,0 +1,43 @@
+----------------------------------------------------------------------
+----------------------------------------------------------------------
+--
+-- String module extension
+--
+----------------------------------------------------------------------
+----------------------------------------------------------------------
+
+-- Courtesy of lua-users.org
+function string.split(str, pat)
+   local t = {} 
+   local fpat = "(.-)" .. pat
+   local last_end = 1
+   local s, e, cap = string.find(str, fpat, 1)
+   while s do
+      if s ~= 1 or cap ~= "" then
+          table.insert(t,cap)
+       end
+      last_end = e+1
+      s, e, cap = string.find(str, fpat, last_end)
+   end
+   if last_end <= string.len(str) then
+      cap = string.sub(str, last_end)
+      table.insert(t, cap)
+   end
+   return t
+end
+
+-- "match" is regularly used as a keyword for pattern matching, 
+-- so here is an always available substitute.
+string.strmatch = string["match"]
+
+-- change a compiled string into a function
+function string.undump(str)
+   if str:strmatch '^\027LuaQ' or str:strmatch '^#![^\n]+\n\027LuaQ' then
+      local f = (lua_loadstring or loadstring)(str)
+      return f
+   else
+      error "Not a chunk dump"
+   end
+end
+
+return string

lualibs/metalua/metalua/table2.lua

@@ -0,0 +1,380 @@
+---------------------------------------------------------------------
+----------------------------------------------------------------------
+--
+-- Table module extension
+--
+----------------------------------------------------------------------
+----------------------------------------------------------------------
+
+-- todo: table.scan (scan1?) fold1? flip?
+
+function table.transpose(t)
+   local tt = { }
+   for a, b in pairs(t) do tt[b] = a end
+   return tt
+end
+
+function table.iforeach(f, ...)
+   -- assert (type (f) == "function") [wouldn't allow metamethod __call]
+   local nargs = select("#", ...)
+   if nargs==1 then -- Quick iforeach (most common case), just one table arg
+      local t = ...
+      assert (type (t) == "table")
+      for i = 1, #t do 
+         local result = f (t[i])
+         -- If the function returns non-false, stop iteration
+         if result then return result end
+      end
+   else -- advanced case: boundaries and/or multiple tables
+
+      -- fargs:       arguments fot a single call to f
+      -- first, last: indexes of the first & last elements mapped in each table
+      -- arg1:        index of the first table in args
+
+      -- 1 - find boundaries if any
+      local  args, fargs, first, last, arg1 = {...}, { }
+      if     type(args[1]) ~= "number" then first, arg1 = 1, 1 -- no boundary
+      elseif type(args[2]) ~= "number" then first, last, arg1 = 1, args[1], 2
+      else   first,  last, arg1 = args[1], args[2], 3 end
+      assert (nargs >= arg1) -- at least one table
+      -- 2 - determine upper boundary if not given
+      if not last then for i = arg1, nargs do 
+            assert (type (args[i]) == "table")
+            last = max (#args[i], last) 
+      end end
+      -- 3 - remove non-table arguments from args, adjust nargs
+      if arg1>1 then args = { select(arg1, unpack(args)) }; nargs = #args end
+
+      -- 4 - perform the iteration
+      for i = first, last do
+         for j = 1, nargs do fargs[j] = args[j][i] end -- build args list
+         local result = f (unpack (fargs)) -- here is the call
+         -- If the function returns non-false, stop iteration
+         if result then return result end
+      end
+   end
+end
+
+function table.imap (f, ...)
+   local result, idx = { }, 1
+   local function g(...) result[idx] = f(...);  idx=idx+1 end
+   table.iforeach(g, ...)
+   return result
+end
+
+function table.ifold (f, acc, ...)
+   local function g(...) acc = f (acc,...) end
+   table.iforeach (g, ...)
+   return acc
+end
+
+-- function table.ifold1 (f, ...)
+--    return table.ifold (f, acc, 2, false, ...)
+-- end
+
+function table.izip(...)
+   local function g(...) return {...} end
+   return table.imap(g, ...)
+end
+
+function table.ifilter(f, t)
+   local yes, no = { }, { }
+   for i=1,#t do table.insert (f(t[i]) and yes or no, t[i]) end
+   return yes, no
+end
+
+function table.icat(...)
+   local result = { }
+   for t in values {...} do
+      for x in values (t) do
+         table.insert (result, x)
+      end
+   end
+   return result
+end
+
+function table.iflatten (x) return table.icat (unpack (x)) end
+
+function table.irev (t)
+   local result, nt = { }, #t
+   for i=0, nt-1 do result[nt-i] = t[i+1] end
+   return result
+end
+
+function table.isub (t, ...)
+   local ti, u = table.insert, { }
+   local args, nargs = {...}, select("#", ...)
+   for i=1, nargs/2 do
+      local a, b = args[2*i-1], args[2*i]
+      for i=a, b, a<=b and 1 or -1 do ti(u, t[i]) end
+   end
+   return u
+end
+
+function table.iall (f, ...)
+   local result = true
+   local function g(...) return not f(...) end
+   return not table.iforeach(g, ...)
+   --return result
+end
+
+function table.iany (f, ...)
+   local function g(...) return not f(...) end
+   return not table.iall(g, ...)
+end
+
+function table.shallow_copy(x)
+   local y={ }
+   for k, v in pairs(x) do y[k]=v end
+   return y
+end
+
+-- Warning, this is implementation dependent: it relies on
+-- the fact the [next()] enumerates the array-part before the hash-part.
+function table.cat(...)
+   local y={ }
+   for x in values{...} do
+      -- cat array-part
+      for _, v in ipairs(x) do table.insert(y,v) end
+      -- cat hash-part
+      local lx, k = #x
+      if lx>0 then k=next(x,lx) else k=next(x) end
+      while k do y[k]=x[k]; k=next(x,k) end
+   end
+   return y
+end
+
+function table.deep_copy(x) 
+   local tracker = { }
+   local function aux (x)
+      if type(x) == "table" then
+         local y=tracker[x]
+         if y then return y end
+         y = { }; tracker[x] = y
+         setmetatable (y, getmetatable (x))
+         for k,v in pairs(x) do y[aux(k)] = aux(v) end
+         return y
+      else return x end
+   end
+   return aux(x)
+end
+
+function table.override(dst, src)
+   for k, v in pairs(src) do dst[k] = v end
+   for i = #src+1, #dst   do dst[i] = nil end
+   return dst
+end
+
+
+function table.range(a,b,c)
+   if not b then assert(not(c)); b=a; a=1
+   elseif not c then c = (b>=a) and 1 or -1 end
+   local result = { }
+   for i=a, b, c do table.insert(result, i) end
+   return result
+end
+
+-- FIXME: new_indent seems to be always nil?!
+-- FIXME: accumulator function should be configurable,
+-- so that print() doesn't need to bufferize the whole string
+-- before starting to print.
+function table.tostring(t, ...)
+   local PRINT_HASH, HANDLE_TAG, FIX_INDENT, LINE_MAX, INITIAL_INDENT = true, true
+   for _, x in ipairs {...} do
+      if type(x) == "number" then
+         if not LINE_MAX then LINE_MAX = x
+         else INITIAL_INDENT = x end
+      elseif x=="nohash" then PRINT_HASH = false
+      elseif x=="notag"  then HANDLE_TAG = false
+      else
+         local n = string['match'](x, "^indent%s*(%d*)$")
+         if n then FIX_INDENT = tonumber(n) or 3 end
+      end
+   end
+   LINE_MAX       = LINE_MAX or math.huge
+   INITIAL_INDENT = INITIAL_INDENT or 1
+   
+   local current_offset =  0  -- indentation level
+   local xlen_cache     = { } -- cached results for xlen()
+   local acc_list       = { } -- Generated bits of string
+   local function acc(...)    -- Accumulate a bit of string
+      local x = table.concat{...}
+      current_offset = current_offset + #x
+      table.insert(acc_list, x) 
+   end
+   local function valid_id(x)
+      -- FIXME: we should also reject keywords; but the list of
+      -- current keywords is not fixed in metalua...
+      return type(x) == "string" 
+         and string['match'](x, "^[a-zA-Z_][a-zA-Z0-9_]*$")
+   end
+   
+   -- Compute the number of chars it would require to display the table
+   -- on a single line. Helps to decide whether some carriage returns are
+   -- required. Since the size of each sub-table is required many times,
+   -- it's cached in [xlen_cache].
+   local xlen_type = { }
+   local function xlen(x, nested)
+      nested = nested or { }
+      if x==nil then return #"nil" end
+      --if nested[x] then return #tostring(x) end -- already done in table
+      local len = xlen_cache[x]
+      if len then return len end
+      local f = xlen_type[type(x)]
+      if not f then return #tostring(x) end
+      len = f (x, nested) 
+      xlen_cache[x] = len
+      return len
+   end
+
+   -- optim: no need to compute lengths if I'm not going to use them
+   -- anyway.
+   if LINE_MAX == math.huge then xlen = function() return 0 end end
+
+   xlen_type["nil"] = function () return 3 end
+   function xlen_type.number  (x) return #tostring(x) end
+   function xlen_type.boolean (x) return x and 4 or 5 end
+   function xlen_type.string  (x) return #string.format("%q",x) end
+   function xlen_type.table   (adt, nested)
+
+      -- Circular references detection
+      if nested [adt] then return #tostring(adt) end
+      nested [adt] = true
+
+      local has_tag  = HANDLE_TAG and valid_id(adt.tag)
+      local alen     = #adt
+      local has_arr  = alen>0
+      local has_hash = false
+      local x = 0
+      
+      if PRINT_HASH then
+         -- first pass: count hash-part
+         for k, v in pairs(adt) do
+            if k=="tag" and has_tag then 
+               -- this is the tag -> do nothing!
+            elseif type(k)=="number" and k<=alen and math.fmod(k,1)==0 then 
+               -- array-part pair -> do nothing!
+            else
+               has_hash = true
+               if valid_id(k) then x=x+#k
+               else x = x + xlen (k, nested) + 2 end -- count surrounding brackets
+               x = x + xlen (v, nested) + 5          -- count " = " and ", "
+            end
+         end
+      end
+
+      for i = 1, alen do x = x + xlen (adt[i], nested) + 2 end -- count ", "
+      
+      nested[adt] = false -- No more nested calls
+
+      if not (has_tag or has_arr or has_hash) then return 3 end
+      if has_tag then x=x+#adt.tag+1 end
+      if not (has_arr or has_hash) then return x end
+      if not has_hash and alen==1 and type(adt[1])~="table" then
+         return x-2 -- substract extraneous ", "
+      end
+      return x+2 -- count "{ " and " }", substract extraneous ", "
+   end
+   
+   -- Recursively print a (sub) table at given indentation level.
+   -- [newline] indicates whether newlines should be inserted.
+   local function rec (adt, nested, indent)
+      if not FIX_INDENT then indent = current_offset end
+      local function acc_newline()
+         acc ("\n"); acc (string.rep (" ", indent)) 
+         current_offset = indent
+      end
+      local x = { }
+      x["nil"] = function() acc "nil" end
+      function x.number()   acc (tostring (adt)) end
+      --function x.string()   acc (string.format ("%q", adt)) end
+      function x.string()   acc ((string.format ("%q", adt):gsub("\\\n", "\\n"))) end
+      function x.boolean()  acc (adt and "true" or "false") end
+      function x.table()
+         if nested[adt] then acc(tostring(adt)); return end
+         nested[adt]  = true
+
+
+         local has_tag  = HANDLE_TAG and valid_id(adt.tag)
+         local alen     = #adt
+         local has_arr  = alen>0
+         local has_hash = false
+
+         if has_tag then acc("`"); acc(adt.tag) end
+
+         -- First pass: handle hash-part
+         if PRINT_HASH then
+            for k, v in pairs(adt) do
+               -- pass if the key belongs to the array-part or is the "tag" field
+               if not (k=="tag" and HANDLE_TAG) and 
+                  not (type(k)=="number" and k<=alen and math.fmod(k,1)==0) then
+
+                  -- Is it the first time we parse a hash pair?
+                  if not has_hash then 
+                     acc "{ "
+                     if not FIX_INDENT then indent = current_offset end
+                  else acc ", " end
+
+                  -- Determine whether a newline is required
+                  local is_id, expected_len = valid_id(k)
+                  if is_id then expected_len = #k + xlen (v, nested) + #" = , "
+                  else expected_len = xlen (k, nested) + 
+                                      xlen (v, nested) + #"[] = , " end
+                  if has_hash and expected_len + current_offset > LINE_MAX
+                  then acc_newline() end
+                  
+                  -- Print the key
+                  if is_id then acc(k); acc " = " 
+                  else  acc "["; rec (k, nested, indent+(FIX_INDENT or 0)); acc "] = " end
+
+                  -- Print the value
+                  rec (v, nested, indent+(FIX_INDENT or 0))
+                  has_hash = true
+               end
+            end
+         end
+
+         -- Now we know whether there's a hash-part, an array-part, and a tag.
+         -- Tag and hash-part are already printed if they're present.
+         if not has_tag and not has_hash and not has_arr then acc "{ }"; 
+         elseif has_tag and not has_hash and not has_arr then -- nothing, tag already in acc
+         else 
+            assert (has_hash or has_arr)
+            local no_brace = false
+            if has_hash and has_arr then acc ", " 
+            elseif has_tag and not has_hash and alen==1 and type(adt[1])~="table" then
+               -- No brace required; don't print "{", remember not to print "}"
+               acc (" "); rec (adt[1], nested, indent+(FIX_INDENT or 0))
+               no_brace = true
+            elseif not has_hash then
+               -- Braces required, but not opened by hash-part handler yet
+               acc "{ "
+               if not FIX_INDENT then indent = current_offset end
+            end
+
+            -- 2nd pass: array-part
+            if not no_brace and has_arr then 
+               rec (adt[1], nested, indent+(FIX_INDENT or 0))
+               for i=2, alen do 
+                  acc ", ";                   
+                  if   current_offset + xlen (adt[i], { }) > LINE_MAX
+                  then acc_newline() end
+                  rec (adt[i], nested, indent+(FIX_INDENT or 0)) 
+               end
+            end
+            if not no_brace then acc " }" end
+         end
+         nested[adt] = false -- No more nested calls
+      end
+      local y = x[type(adt)]
+      if y then y() else acc(tostring(adt)) end
+   end
+   --printf("INITIAL_INDENT = %i", INITIAL_INDENT)
+   current_offset = INITIAL_INDENT or 0
+   rec(t, { }, 0)
+   return table.concat (acc_list)
+end
+
+function table.print(...) return print(table.tostring(...)) end
+
+return table

lualibs/metalua/mlp_expr.lua

@@ -0,0 +1,213 @@
+----------------------------------------------------------------------
+-- Metalua:  $Id: mlp_expr.lua,v 1.7 2006/11/15 09:07:50 fab13n Exp $
+--
+-- Summary: metalua parser, expression parser. This is part of the
+--   definition of module [mlp].
+--
+----------------------------------------------------------------------
+--
+-- Copyright (c) 2006, Fabien Fleutot <metalua@gmail.com>.
+--
+-- This software is released under the MIT Licence, see licence.txt
+-- for details.
+--
+----------------------------------------------------------------------
+-- History:
+-- $Log: mlp_expr.lua,v $
+-- Revision 1.7  2006/11/15 09:07:50  fab13n
+-- debugged meta operators.
+-- Added command line options handling.
+--
+-- Revision 1.6  2006/11/10 02:11:17  fab13n
+-- compiler faithfulness to 5.1 improved
+-- gg.expr extended
+-- mlp.expr refactored
+--
+-- Revision 1.5  2006/11/09 09:39:57  fab13n
+-- some cleanup
+--
+-- Revision 1.4  2006/11/07 21:29:02  fab13n
+-- improved quasi-quoting
+--
+-- Revision 1.3  2006/11/07 04:38:00  fab13n
+-- first bootstrapping version.
+--
+-- Revision 1.2  2006/11/05 15:08:34  fab13n
+-- updated code generation, to be compliant with 5.1
+--
+----------------------------------------------------------------------
+
+--------------------------------------------------------------------------------
+--
+-- Exported API:
+-- * [mlp.expr()]
+-- * [mlp.expr_list()]
+-- * [mlp.func_val()]
+--
+--------------------------------------------------------------------------------
+
+--require "gg"
+--require "mlp_misc"
+--require "mlp_table"
+--require "mlp_meta"
+
+--------------------------------------------------------------------------------
+-- These function wrappers (eta-expansions ctually) are just here to break
+-- some circular dependencies between mlp_xxx.lua files.
+--------------------------------------------------------------------------------
+local function _expr (lx) return mlp.expr (lx)  end
+local function _table_content (lx) return mlp.table_content (lx) end
+local function block (lx) return mlp.block (lx) end
+local function stat  (lx) return mlp.stat (lx)  end
+
+module ("mlp", package.seeall)
+
+--------------------------------------------------------------------------------
+-- Non-empty expression list. Actually, this isn't used here, but that's
+-- handy to give to users.
+--------------------------------------------------------------------------------
+expr_list = gg.list{ _expr, separators = "," }
+
+--------------------------------------------------------------------------------
+-- Helpers for function applications / method applications
+--------------------------------------------------------------------------------
+func_args_content = gg.list { 
+   name = "function arguments",
+   _expr, separators = ",", terminators = ")" } 
+
+-- Used to parse methods
+method_args = gg.multisequence{
+   name = "function argument(s)",
+   { "{", table_content, "}" },
+   { "(", func_args_content, ")", builder = fget(1) },
+   { "+{", quote_content, "}" }, 
+   function(lx) local r = opt_string(lx); return r and {r} or { } end }
+
+--------------------------------------------------------------------------------
+-- [func_val] parses a function, from opening parameters parenthese to
+-- "end" keyword included. Used for anonymous functions as well as
+-- function declaration statements (both local and global).
+--
+-- It's wrapped in a [_func_val] eta expansion, so that when expr
+-- parser uses the latter, they will notice updates of [func_val]
+-- definitions.
+--------------------------------------------------------------------------------
+func_params_content = gg.list{ name="function parameters",
+   gg.multisequence{ { "...", builder = "Dots" }, id },
+   separators  = ",", terminators = {")", "|"} } 
+
+local _func_params_content = function (lx) return func_params_content(lx) end
+
+func_val = gg.sequence { name="function body",
+   "(", func_params_content, ")", block, "end", builder = "Function" }
+
+local _func_val = function (lx) return func_val(lx) end
+
+--------------------------------------------------------------------------------
+-- Default parser for primary expressions
+--------------------------------------------------------------------------------
+function id_or_literal (lx)
+   local a = lx:next()
+   if a.tag~="Id" and a.tag~="String" and a.tag~="Number" then
+      local msg
+      if a.tag=='Eof' then
+         msg = "End of file reached when an expression was expected"
+      elseif a.tag=='Keyword' then
+         msg = "An expression was expected, and `"..a[1]..
+            "' can't start an expression"
+      else
+         msg = "Unexpected expr token " .. _G.table.tostring (a, 'nohash')
+      end
+      gg.parse_error (lx, msg)
+   end
+   return a
+end
+
+
+--------------------------------------------------------------------------------
+-- Builder generator for operators. Wouldn't be worth it if "|x|" notation
+-- were allowed, but then lua 5.1 wouldn't compile it 
+--------------------------------------------------------------------------------
+
+-- opf1 = |op| |_,a| `Op{ op, a }
+local function opf1 (op) return 
+   function (_,a) return { tag="Op", op, a } end end
+
+-- opf2 = |op| |a,_,b| `Op{ op, a, b }
+local function opf2 (op) return 
+   function (a,_,b) return { tag="Op", op, a, b } end end
+
+-- opf2r = |op| |a,_,b| `Op{ op, b, a } -- (args reversed)
+local function opf2r (op) return 
+   function (a,_,b) return { tag="Op", op, b, a } end end
+
+local function op_ne(a, _, b) 
+   -- The first version guarantees to return the same code as Lua,
+   -- but it relies on the non-standard 'ne' operator, which has been
+   -- suppressed from the official AST grammar (although still supported
+   -- in practice by the compiler).
+   -- return { tag="Op", "ne", a, b }
+   return { tag="Op", "not", { tag="Op", "eq", a, b, lineinfo= {
+            first = a.lineinfo.first, last = b.lineinfo.last } } }
+end
+   
+
+--------------------------------------------------------------------------------
+--
+-- complete expression
+--
+--------------------------------------------------------------------------------
+
+-- FIXME: set line number. In [expr] transformers probably
+
+expr = gg.expr { name = "expression",
+
+   primary = gg.multisequence{ name="expr primary",
+      { "(", _expr, ")",           builder = "Paren" },
+      { "function", _func_val,     builder = fget(1) },
+      { "-{", splice_content, "}", builder = fget(1) },
+      { "+{", quote_content, "}",  builder = fget(1) }, 
+      { "nil",                     builder = "Nil" },
+      { "true",                    builder = "True" },
+      { "false",                   builder = "False" },
+      { "...",                     builder = "Dots" },
+      table,
+      id_or_literal },
+
+   infix = { name="expr infix op",
+      { "+",  prec = 60, builder = opf2 "add"  },
+      { "-",  prec = 60, builder = opf2 "sub"  },
+      { "*",  prec = 70, builder = opf2 "mul"  },
+      { "/",  prec = 70, builder = opf2 "div"  },
+      { "%",  prec = 70, builder = opf2 "mod"  },
+      { "^",  prec = 90, builder = opf2 "pow",    assoc = "right" },
+      { "..", prec = 40, builder = opf2 "concat", assoc = "right" },
+      { "==", prec = 30, builder = opf2 "eq"  },
+      { "~=", prec = 30, builder = op_ne  },
+      { "<",  prec = 30, builder = opf2 "lt"  },
+      { "<=", prec = 30, builder = opf2 "le"  },
+      { ">",  prec = 30, builder = opf2r "lt"  },
+      { ">=", prec = 30, builder = opf2r "le"  },
+      { "and",prec = 20, builder = opf2 "and" },
+      { "or", prec = 10, builder = opf2 "or"  } },
+
+   prefix = { name="expr prefix op",
+      { "not", prec = 80, builder = opf1 "not" },
+      { "#",   prec = 80, builder = opf1 "len" },
+      { "-",   prec = 80, builder = opf1 "unm" } },
+
+   suffix = { name="expr suffix op",
+      { "[", _expr, "]", builder = function (tab, idx) 
+         return {tag="Index", tab, idx[1]} end},
+      { ".", id, builder = function (tab, field) 
+         return {tag="Index", tab, id2string(field[1])} end },
+      { "(", func_args_content, ")", builder = function(f, args) 
+         return {tag="Call", f, unpack(args[1])} end },
+      { "{", _table_content, "}", builder = function (f, arg)
+         return {tag="Call", f, arg[1]} end},
+      { ":", id, method_args, builder = function (obj, post)
+         return {tag="Invoke", obj, id2string(post[1]), unpack(post[2])} end},
+      { "+{", quote_content, "}", builder = function (f, arg) 
+         return {tag="Call", f,  arg[1] } end },
+      default = { name="opt_string_arg", parse = mlp.opt_string, builder = function(f, arg) 
+         return {tag="Call", f, arg } end } } }

lualibs/metalua/mlp_ext.lua

@@ -0,0 +1,89 @@
+--------------------------------------------------------------------------------
+--
+-- Non-Lua syntax extensions
+--
+--------------------------------------------------------------------------------
+
+module ("mlp", package.seeall)
+
+--------------------------------------------------------------------------------
+-- Alebraic Datatypes
+--------------------------------------------------------------------------------
+local function adt (lx)
+   local tagval = id (lx) [1]
+   local tagkey = {tag="Pair", {tag="String", "tag"}, {tag="String", tagval} }
+   if lx:peek().tag == "String" or lx:peek().tag == "Number" then
+      return { tag="Table", tagkey, lx:next() }
+   elseif lx:is_keyword (lx:peek(), "{") then
+      local x = table (lx)
+      _G.table.insert (x, 1, tagkey)
+      return x
+   else return { tag="Table", tagkey } end
+end
+
+expr:add{ "`", adt, builder = fget(1) }
+
+--------------------------------------------------------------------------------
+-- Anonymous lambda
+--------------------------------------------------------------------------------
+local lambda_expr = gg.sequence{ 
+   "|", func_params_content, "|", expr,
+   builder= function (x) 
+      local li = x[2].lineinfo
+      return { tag="Function", x[1], 
+               { {tag="Return", x[2], lineinfo=li }, lineinfo=li } }
+   end }
+
+-- In an earlier version, lambda_expr took an expr_list rather than an expr
+-- after the 2nd bar. However, it happened to be much more of a burden than an
+-- help, So finally I disabled it. If you want to return several results, 
+-- use the long syntax.
+--------------------------------------------------------------------------------
+-- local lambda_expr = gg.sequence{ 
+--    "|", func_params_content, "|", expr_list,
+--    builder= function (x) 
+--       return {tag="Function", x[1], { {tag="Return", unpack(x[2]) } } } end }
+
+expr:add (lambda_expr)
+
+--------------------------------------------------------------------------------
+-- Allows to write "a `f` b" instead of "f(a, b)". Taken from Haskell.
+-- This is not part of Lua 5.1 syntax, so it's added to the expression
+-- afterwards, so that it's easier to disable.
+--------------------------------------------------------------------------------
+local function expr_in_backquotes (lx) return expr(lx, 35) end
+
+expr.infix:add{ name = "infix function", 
+   "`", expr_in_backquotes, "`", prec = 35, assoc="left", 
+   builder = function(a, op, b) return {tag="Call", op[1], a, b} end }
+
+
+--------------------------------------------------------------------------------
+-- table.override assignment
+--------------------------------------------------------------------------------
+
+mlp.lexer:add "<-"
+stat.assignments["<-"] = function (a, b)
+   assert( #a==1 and #b==1, "No multi-args for '<-'")                         
+   return { tag="Call", { tag="Index", { tag="Id", "table" },
+                                       { tag="String", "override" } },
+                        a[1], b[1]} 
+end
+
+--------------------------------------------------------------------------------
+-- C-style op+assignments
+--------------------------------------------------------------------------------
+local function op_assign(kw, op) 
+   local function rhs(a, b)
+      return { tag="Op", op, a, b } 
+   end
+   local function f(a,b) 
+      return { tag="Set", a, _G.table.imap(rhs, a, b) }
+   end
+   mlp.lexer:add (kw)
+   mlp.stat.assignments[kw] = f
+end
+
+_G.table.iforeach (op_assign, 
+                {"+=", "-=", "*=", "/="},
+                {"add", "sub", "mul", "div"})

lualibs/metalua/mlp_lexer.lua

@@ -0,0 +1,32 @@
+----------------------------------------------------------------------
+-- Metalua:  $Id: mll.lua,v 1.3 2006/11/15 09:07:50 fab13n Exp $
+--
+-- Summary: Source file lexer. ~~Currently only works on strings.
+-- Some API refactoring is needed.
+--
+----------------------------------------------------------------------
+--
+-- Copyright (c) 2006-2007, Fabien Fleutot <metalua@gmail.com>.
+--
+-- This software is released under the MIT Licence, see licence.txt
+-- for details.
+--
+----------------------------------------------------------------------
+
+module ("mlp", package.seeall)
+
+require "lexer"
+
+local mlp_lexer = lexer.lexer:clone()
+
+local keywords = {
+    "and", "break", "do", "else", "elseif",
+    "end", "false", "for", "function", "if",
+    "in", "local", "nil", "not", "or", "repeat",
+    "return", "then", "true", "until", "while",
+    "...", "..", "==", ">=", "<=", "~=", 
+    "+{", "-{" }
+ 
+for _,w in pairs(keywords) do mlp_lexer:add(w) end -- PK 6/4/2012
+
+_M.lexer = mlp_lexer

lualibs/metalua/mlp_meta.lua

@@ -0,0 +1,118 @@
+----------------------------------------------------------------------
+-- Metalua:  $Id: mlp_meta.lua,v 1.4 2006/11/15 09:07:50 fab13n Exp $
+--
+-- Summary: Meta-operations: AST quasi-quoting and splicing
+--
+----------------------------------------------------------------------
+--
+-- Copyright (c) 2006, Fabien Fleutot <metalua@gmail.com>.
+--
+-- This software is released under the MIT Licence, see licence.txt
+-- for details.
+--
+----------------------------------------------------------------------
+
+
+--------------------------------------------------------------------------------
+--
+-- Exported API:
+-- * [mlp.splice_content()]
+-- * [mlp.quote_content()]
+--
+--------------------------------------------------------------------------------
+
+module ("mlp", package.seeall)
+
+--------------------------------------------------------------------------------
+-- External splicing: compile an AST into a chunk, load and evaluate
+-- that chunk, and replace the chunk by its result (which must also be
+-- an AST).
+--------------------------------------------------------------------------------
+
+function splice (ast)
+   local f = mlc.function_of_ast(ast, '=splice')
+   local result=f()
+   return result
+end
+
+--------------------------------------------------------------------------------
+-- Going from an AST to an AST representing that AST
+-- the only key being lifted in this version is ["tag"]
+--------------------------------------------------------------------------------
+function quote (t)
+   --print("QUOTING:", _G.table.tostring(t, 60))
+   local cases = { }
+   function cases.table (t)
+      local mt = { tag = "Table" }
+      --_G.table.insert (mt, { tag = "Pair", quote "quote", { tag = "True" } })
+      if t.tag == "Splice" then
+         assert (#t==1, "Invalid splice")
+         local sp = t[1]
+         return sp
+      elseif t.tag then
+         _G.table.insert (mt, { tag = "Pair", quote "tag", quote (t.tag) })
+      end
+      for _, v in ipairs (t) do
+         _G.table.insert (mt, quote(v))
+      end
+      return mt
+   end
+   function cases.number (t) return { tag = "Number", t, quote = true } end
+   function cases.string (t) return { tag = "String", t, quote = true } end
+   return cases [ type (t) ] (t)
+end
+
+--------------------------------------------------------------------------------
+-- when this variable is false, code inside [-{...}] is compiled and
+-- avaluated immediately. When it's true (supposedly when we're
+-- parsing data inside a quasiquote), [-{foo}] is replaced by
+-- [`Splice{foo}], which will be unpacked by [quote()].
+--------------------------------------------------------------------------------
+in_a_quote = false
+
+--------------------------------------------------------------------------------
+-- Parse the inside of a "-{ ... }"
+--------------------------------------------------------------------------------
+function splice_content (lx)
+   local parser_name = "expr"
+   if lx:is_keyword (lx:peek(2), ":") then
+      local a = lx:next()
+      lx:next() -- skip ":"
+      assert (a.tag=="Id", "Invalid splice parser name")
+      parser_name = a[1]
+   end
+   local ast = mlp[parser_name](lx)
+   if in_a_quote then
+      --printf("SPLICE_IN_QUOTE:\n%s", _G.table.tostring(ast, "nohash", 60))
+      return { tag="Splice", ast }
+   else
+      if parser_name == "expr" then ast = { { tag="Return", ast } }
+      elseif parser_name == "stat"  then ast = { ast }
+      elseif parser_name ~= "block" then
+         error ("splice content must be an expr, stat or block") end
+      --printf("EXEC THIS SPLICE:\n%s", _G.table.tostring(ast, "nohash", 60))
+      return splice (ast)
+   end
+end
+
+--------------------------------------------------------------------------------
+-- Parse the inside of a "+{ ... }"
+--------------------------------------------------------------------------------
+function quote_content (lx)
+   local parser 
+   if lx:is_keyword (lx:peek(2), ":") then -- +{parser: content }
+      parser = mlp[id(lx)[1]]
+      lx:next()
+   else -- +{ content }
+      parser = mlp.expr
+   end
+
+   local prev_iq = in_a_quote
+   in_a_quote = true
+   --print("IN_A_QUOTE")
+   local content = parser (lx)
+   local q_content = quote (content)
+   in_a_quote = prev_iq
+   return q_content
+end
+

lualibs/metalua/mlp_misc.lua

@@ -0,0 +1,185 @@
+----------------------------------------------------------------------
+-- Metalua:  $Id: mlp_misc.lua,v 1.6 2006/11/15 09:07:50 fab13n Exp $
+--
+-- Summary: metalua parser, miscellaneous utility functions.
+--
+----------------------------------------------------------------------
+--
+-- Copyright (c) 2006, Fabien Fleutot <metalua@gmail.com>.
+--
+-- This software is released under the MIT Licence, see licence.txt
+-- for details.
+--
+----------------------------------------------------------------------
+-- History:
+-- $Log: mlp_misc.lua,v $
+-- Revision 1.6  2006/11/15 09:07:50  fab13n
+-- debugged meta operators.
+-- Added command line options handling.
+--
+-- Revision 1.5  2006/11/10 02:11:17  fab13n
+-- compiler faithfulness to 5.1 improved
+-- gg.expr extended
+-- mlp.expr refactored
+--
+-- Revision 1.4  2006/11/09 09:39:57  fab13n
+-- some cleanup
+--
+-- Revision 1.3  2006/11/07 04:38:00  fab13n
+-- first bootstrapping version.
+--
+-- Revision 1.2  2006/11/05 15:08:34  fab13n
+-- updated code generation, to be compliant with 5.1
+--
+----------------------------------------------------------------------
+
+--------------------------------------------------------------------------------
+--
+-- Exported API:
+-- * [mlp.fget()]
+-- * [mlp.id()]
+-- * [mlp.opt_id()]
+-- * [mlp.id_list()]
+-- * [mlp.gensym()]
+-- * [mlp.string()]
+-- * [mlp.opt_string()]
+-- * [mlp.id2string()]
+--
+--------------------------------------------------------------------------------
+
+--require "gg"
+--require "mll"
+
+module ("mlp", package.seeall)
+
+--------------------------------------------------------------------------------
+-- returns a function that takes the [n]th element of a table.
+-- if [tag] is provided, then this element is expected to be a
+-- table, and this table receives a "tag" field whose value is
+-- set to [tag].
+--
+-- The primary purpose of this is to generate builders for
+-- grammar generators. It has little purpose in metalua, as lambda has
+-- a lightweight syntax.
+--------------------------------------------------------------------------------
+
+function fget (n, tag) 
+   assert (type (n) == "number")
+   if tag then
+      assert (type (tag) == "string")
+      return function (x) 
+         assert (type (x[n]) == "table")       
+         return {tag=tag, unpack(x[n])} end 
+   else
+      return function (x) return x[n] end 
+   end
+end
+
+
+--------------------------------------------------------------------------------
+-- Try to read an identifier (possibly as a splice), or return [false] if no
+-- id is found.
+--------------------------------------------------------------------------------
+function opt_id (lx)
+   local a = lx:peek();
+   if lx:is_keyword (a, "-{") then
+      local v = gg.sequence{ "-{", splice_content, "}" } (lx) [1]
+      if v.tag ~= "Id" and v.tag ~= "Splice" then
+         gg.parse_error(lx,"Bad id splice")
+      end
+      return v
+   elseif a.tag == "Id" then return lx:next()
+   else return false end
+end
+
+--------------------------------------------------------------------------------
+-- Mandatory reading of an id: causes an error if it can't read one.
+--------------------------------------------------------------------------------
+function id (lx)
+   return opt_id (lx) or gg.parse_error(lx,"Identifier expected")
+end
+
+--------------------------------------------------------------------------------
+-- Common helper function
+--------------------------------------------------------------------------------
+id_list = gg.list { primary = mlp.id, separators = "," }
+
+--------------------------------------------------------------------------------
+-- Symbol generator: [gensym()] returns a guaranteed-to-be-unique identifier.
+-- The main purpose is to avoid variable capture in macros.
+--
+-- If a string is passed as an argument, theis string will be part of the
+-- id name (helpful for macro debugging)
+--------------------------------------------------------------------------------
+local gensymidx = 0
+
+function gensym (arg)
+   gensymidx = gensymidx + 1
+   return { tag="Id", _G.string.format(".%i.%s", gensymidx, arg or "")}
+end
+
+--------------------------------------------------------------------------------
+-- Converts an identifier into a string. Hopefully one day it'll handle
+-- splices gracefully, but that proves quite tricky.
+--------------------------------------------------------------------------------
+function id2string (id)
+   --print("id2string:", disp.ast(id))
+   if id.tag == "Id" then id.tag = "String"; return id
+   elseif id.tag == "Splice" then
+      assert (in_a_quote, "can't do id2string on an outermost splice")
+      error ("id2string on splice not implemented")
+      -- Evaluating id[1] will produce `Id{ xxx },
+      -- and we want it to produce `String{ xxx }
+      -- Morally, this is what I want:
+      -- return `String{ `Index{ `Splice{ id[1] }, `Number 1 } }
+      -- That is, without sugar:
+      return {tag="String",  {tag="Index", {tag="Splice", id[1] }, 
+                                           {tag="Number", 1 } } }
+   else error ("Identifier expected: ".._G.table.tostring(id, 'nohash')) end
+end
+
+--------------------------------------------------------------------------------
+-- Read a string, possibly spliced, or return an error if it can't
+--------------------------------------------------------------------------------
+function string (lx)
+   local a = lx:peek()
+   if lx:is_keyword (a, "-{") then
+      local v = gg.sequence{ "-{", splice_content, "}" } (lx) [1]
+      if v.tag ~= "" and v.tag ~= "Splice" then
+         gg.parse_error(lx,"Bad string splice")
+      end
+      return v
+   elseif a.tag == "String" then return lx:next()
+   else error "String expected" end
+end
+
+--------------------------------------------------------------------------------
+-- Try to read a string, or return false if it can't. No splice allowed.
+--------------------------------------------------------------------------------
+function opt_string (lx)
+   return lx:peek().tag == "String" and lx:next()
+end
+   
+--------------------------------------------------------------------------------
+-- Chunk reader: block + Eof
+--------------------------------------------------------------------------------
+function skip_initial_sharp_comment (lx)
+   -- Dirty hack: I'm happily fondling lexer's private parts
+   -- FIXME: redundant with lexer:newstream()
+   lx :sync()
+   local i = lx.src:match ("^#.-\n()", lx.i)
+   if i then lx.i, lx.column_offset, lx.line = i, i, lx.line+1 end
+end
+
+local function _chunk (lx)
+   if lx:peek().tag == 'Eof' then return { } -- handle empty files
+   else 
+      skip_initial_sharp_comment (lx)
+      local chunk = block (lx)
+      if lx:peek().tag ~= "Eof" then error "End-of-file expected" end
+      return chunk
+   end
+end
+
+-- chunk is wrapped in a sequence so that it has a "transformer" field.
+chunk = gg.sequence { _chunk, builder = unpack }

lualibs/metalua/mlp_stat.lua

@@ -0,0 +1,226 @@
+----------------------------------------------------------------------
+-- Metalua:  $Id: mlp_stat.lua,v 1.7 2006/11/15 09:07:50 fab13n Exp $
+--
+-- Summary: metalua parser, statement/block parser. This is part of
+--   the definition of module [mlp].
+--
+----------------------------------------------------------------------
+--
+-- Copyright (c) 2006, Fabien Fleutot <metalua@gmail.com>.
+--
+-- This software is released under the MIT Licence, see licence.txt
+-- for details.
+--
+----------------------------------------------------------------------
+--
+----------------------------------------------------------------------
+
+--------------------------------------------------------------------------------
+--
+-- Exports API:
+-- * [mlp.stat()]
+-- * [mlp.block()]
+-- * [mlp.for_header()]
+--
+--------------------------------------------------------------------------------
+
+--------------------------------------------------------------------------------
+-- eta-expansions to break circular dependency
+--------------------------------------------------------------------------------
+local expr      = function (lx) return mlp.expr     (lx) end
+local func_val  = function (lx) return mlp.func_val (lx) end
+local expr_list = function (lx) return mlp.expr_list(lx) end
+
+module ("mlp", package.seeall)
+
+--------------------------------------------------------------------------------
+-- List of all keywords that indicate the end of a statement block. Users are
+-- likely to extend this list when designing extensions.
+--------------------------------------------------------------------------------
+
+
+local block_terminators = { "else", "elseif", "end", "until", ")", "}", "]" }
+
+-- FIXME: this must be handled from within GG!!!
+function block_terminators:add(x) 
+   if type (x) == "table" then for _, y in ipairs(x) do self:add (y) end
+   else _G.table.insert (self, x) end
+end
+
+--------------------------------------------------------------------------------
+-- list of statements, possibly followed by semicolons
+--------------------------------------------------------------------------------
+block = gg.list {
+   name        = "statements block",
+   terminators = block_terminators,
+   primary     = function (lx)
+      -- FIXME use gg.optkeyword()
+      local x = stat (lx)
+      if lx:is_keyword (lx:peek(), ";") then lx:next() end
+      return x
+   end }
+
+--------------------------------------------------------------------------------
+-- Helper function for "return <expr_list>" parsing.
+-- Called when parsing return statements.
+-- The specific test for initial ";" is because it's not a block terminator,
+-- so without itgg.list would choke on "return ;" statements.
+-- We don't make a modified copy of block_terminators because this list
+-- is sometimes modified at runtime, and the return parser would get out of
+-- sync if it was relying on a copy.
+--------------------------------------------------------------------------------
+local return_expr_list_parser = gg.multisequence{
+   { ";" , builder = function() return { } end }, 
+   default = gg.list { 
+      expr, separators = ",", terminators = block_terminators } }
+
+--------------------------------------------------------------------------------
+-- for header, between [for] and [do] (exclusive).
+-- Return the `Forxxx{...} AST, without the body element (the last one).
+--------------------------------------------------------------------------------
+function for_header (lx)
+   local var = mlp.id (lx)
+   if lx:is_keyword (lx:peek(), "=") then 
+      -- Fornum: only 1 variable
+      lx:next() -- skip "="
+      local e = expr_list (lx)
+      assert (2 <= #e and #e <= 3, "2 or 3 values in a fornum")
+      return { tag="Fornum", var, unpack (e) }
+   else
+      -- Forin: there might be several vars
+      local a = lx:is_keyword (lx:next(), ",", "in")
+      if a=="in" then var_list = { var, lineinfo = var.lineinfo } else
+         -- several vars; first "," skipped, read other vars
+         var_list = gg.list{ 
+            primary = id, separators = ",", terminators = "in" } (lx)
+         _G.table.insert (var_list, 1, var) -- put back the first variable
+         lx:next() -- skip "in"
+      end
+      local e = expr_list (lx)
+      return { tag="Forin", var_list, e }
+   end
+end
+
+--------------------------------------------------------------------------------
+-- Function def parser helper: id ( . id ) *
+--------------------------------------------------------------------------------
+local function fn_builder (list)
+   local r = list[1]
+   for i = 2, #list do r = { tag="Index", r, id2string(list[i]) } end
+   return r
+end
+local func_name = gg.list{ id, separators = ".", builder = fn_builder }
+
+--------------------------------------------------------------------------------
+-- Function def parser helper: ( : id )?
+--------------------------------------------------------------------------------
+local method_name = gg.onkeyword{ name = "method invocation", ":", id, 
+   transformers = { function(x) return x and id2string(x) end } }
+
+--------------------------------------------------------------------------------
+-- Function def builder
+--------------------------------------------------------------------------------
+local function funcdef_builder(x)
+   local name, method, func = x[1], x[2], x[3]
+   if method then 
+      name = { tag="Index", name, method, lineinfo = {
+         first = name.lineinfo.first,
+         last  = method.lineinfo.last } }
+      _G.table.insert (func[1], 1, {tag="Id", "self"}) 
+   end
+   local r = { tag="Set", {name}, {func} } 
+   r[1].lineinfo = name.lineinfo
+   r[2].lineinfo = func.lineinfo
+   return r
+end 
+
+
+--------------------------------------------------------------------------------
+-- if statement builder
+--------------------------------------------------------------------------------
+local function if_builder (x)
+   local cb_pairs, else_block, r = x[1], x[2], {tag="If"}
+   for i=1,#cb_pairs do r[2*i-1]=cb_pairs[i][1]; r[2*i]=cb_pairs[i][2] end
+   if else_block then r[#r+1] = else_block end
+   return r
+end 
+
+--------------------------------------------------------------------------------
+-- produce a list of (expr,block) pairs
+--------------------------------------------------------------------------------
+local elseifs_parser = gg.list {
+   gg.sequence { expr, "then", block },
+   separators  = "elseif",
+   terminators = { "else", "end" } }
+
+--------------------------------------------------------------------------------
+-- assignments and calls: statements that don't start with a keyword
+--------------------------------------------------------------------------------
+local function assign_or_call_stat_parser (lx)
+   local e = expr_list (lx)
+   local a = lx:is_keyword(lx:peek())
+   local op = a and stat.assignments[a]
+   if op then
+      --FIXME: check that [e] is a LHS
+      lx:next()
+      local v = expr_list (lx)
+      if type(op)=="string" then return { tag=op, e, v }
+      else return op (e, v) end
+   else 
+      assert (#e > 0)
+      if #e > 1 then 
+         gg.parse_error (lx, 
+            "comma is not a valid statement separator; statement can be "..
+            "separated by semicolons, or not separated at all") end
+      if e[1].tag ~= "Call" and e[1].tag ~= "Invoke" then
+         local typename
+         if e[1].tag == 'Id' then 
+            typename = '("'..e[1][1]..'") is an identifier'
+         elseif e[1].tag == 'Op' then 
+            typename = "is an arithmetic operation"
+         else typename = "is of type '"..(e[1].tag or "<list>").."'" end
+
+         gg.parse_error (lx, "This expression " .. typename ..
+            "; a statement was expected, and only function and method call "..
+            "expressions can be used as statements");
+      end
+      return e[1]
+   end
+end
+
+local_stat_parser = gg.multisequence{
+   -- local function <name> <func_val>
+   { "function", id, func_val, builder = 
+      function(x) 
+         local vars = { x[1], lineinfo = x[1].lineinfo }
+         local vals = { x[2], lineinfo = x[2].lineinfo }
+         return { tag="Localrec", vars, vals } 
+      end },
+   -- local <id_list> ( = <expr_list> )?
+   default = gg.sequence{ id_list, gg.onkeyword{ "=", expr_list },
+      builder = function(x) return {tag="Local", x[1], x[2] or { } } end } }
+
+--------------------------------------------------------------------------------
+-- statement
+--------------------------------------------------------------------------------
+stat = gg.multisequence { 
+   name="statement",
+   { "do", block, "end", builder = 
+      function (x) return { tag="Do", unpack (x[1]) } end },
+   { "for", for_header, "do", block, "end", builder = 
+      function (x) x[1][#x[1]+1] = x[2]; return x[1] end },
+   { "function", func_name, method_name, func_val, builder=funcdef_builder },
+   { "while", expr, "do", block, "end", builder = "While" },
+   { "repeat", block, "until", expr, builder = "Repeat" },
+   { "local", local_stat_parser, builder = fget (1) },
+   { "return", return_expr_list_parser, builder = fget (1, "Return") },
+   { "break", builder = function() return { tag="Break" } end },
+   { "-{", splice_content, "}", builder = fget(1) },
+   { "if", elseifs_parser, gg.onkeyword{ "else", block }, "end", 
+     builder = if_builder },
+   default = assign_or_call_stat_parser }
+
+stat.assignments = {
+   ["="] = "Set" }
+
+function stat.assignments:add(k, v) self[k] = v end

lualibs/metalua/mlp_table.lua

@@ -0,0 +1,92 @@
+----------------------------------------------------------------------
+-- Metalua:  $Id: mlp_table.lua,v 1.5 2006/11/10 02:11:17 fab13n Exp $
+--
+-- Summary: metalua parser, table constructor parser. This is part 
+--   of thedefinition of module [mlp].
+--
+----------------------------------------------------------------------
+--
+-- Copyright (c) 2006, Fabien Fleutot <metalua@gmail.com>.
+--
+-- This software is released under the MIT Licence, see licence.txt
+-- for details.
+--
+----------------------------------------------------------------------
+-- History:
+-- $Log: mlp_table.lua,v $
+-- Revision 1.5  2006/11/10 02:11:17  fab13n
+-- compiler faithfulness to 5.1 improved
+-- gg.expr extended
+-- mlp.expr refactored
+--
+-- Revision 1.4  2006/11/09 09:39:57  fab13n
+-- some cleanup
+--
+-- Revision 1.3  2006/11/07 04:38:00  fab13n
+-- first bootstrapping version.
+--
+-- Revision 1.2  2006/11/05 15:08:34  fab13n
+-- updated code generation, to be compliant with 5.1
+--
+----------------------------------------------------------------------
+
+--------------------------------------------------------------------------------
+--
+-- Exported API:
+-- * [mlp.table_field()]
+-- * [mlp.table_content()]
+-- * [mlp.table()]
+--
+-- KNOWN BUG: doesn't handle final ";" or "," before final "}"
+--
+--------------------------------------------------------------------------------
+
+--require "gg"
+--require "mll"
+--require "mlp_misc"
+
+module ("mlp", package.seeall)
+
+--------------------------------------------------------------------------------
+-- eta expansion to break circular dependencies:
+--------------------------------------------------------------------------------
+local function _expr (lx) return expr(lx) end
+
+--------------------------------------------------------------------------------
+-- [[key] = value] table field definition
+--------------------------------------------------------------------------------
+local bracket_field = gg.sequence{ "[", _expr, "]", "=", _expr, builder = "Pair" }
+
+--------------------------------------------------------------------------------
+-- [id = value] or [value] table field definition;
+-- [[key]=val] are delegated to [bracket_field()]
+--------------------------------------------------------------------------------
+function table_field (lx)
+   if lx:is_keyword (lx:peek(), "[") then return bracket_field (lx) end
+   local e = _expr (lx)
+   if lx:is_keyword (lx:peek(), "=") then 
+      lx:next(); -- skip the "="
+      local key = id2string(e)
+      local val = _expr(lx)
+      local r = { tag="Pair", key, val } 
+      r.lineinfo = { first = key.lineinfo.first, last = val.lineinfo.last }
+      return r
+   else return e end
+end
+
+local function _table_field(lx) return table_field(lx) end
+
+--------------------------------------------------------------------------------
+-- table constructor, without enclosing braces; returns a full table object
+--------------------------------------------------------------------------------
+table_content = gg.list { _table_field, 
+   separators = { ",", ";" }, terminators = "}", builder = "Table" }
+
+local function _table_content(lx) return table_content(lx) end
+
+--------------------------------------------------------------------------------
+-- complete table constructor including [{...}]
+--------------------------------------------------------------------------------
+table = gg.sequence{ "{", _table_content, "}", builder = fget(1) }
+
+

lualibs/ssl.lua

@@ -0,0 +1,93 @@
+------------------------------------------------------------------------------
+-- LuaSec 0.4.1
+-- Copyright (C) 2006-2011 Bruno Silvestre
+--
+------------------------------------------------------------------------------
+
+module("ssl", package.seeall)
+
+require("ssl.core")
+require("ssl.context")
+
+
+_VERSION   = "0.4.1"
+_COPYRIGHT = "LuaSec 0.4.1 - Copyright (C) 2006-2011 Bruno Silvestre\n" .. 
+             "LuaSocket 2.0.2 - Copyright (C) 2004-2007 Diego Nehab"
+
+-- Export functions
+rawconnection = core.rawconnection
+rawcontext    = context.rawcontext
+
+--
+--
+--
+local function optexec(func, param, ctx)
+  if param then
+    if type(param) == "table" then
+      return func(ctx, unpack(param))
+    else
+      return func(ctx, param)
+    end
+  end
+  return true
+end
+
+--
+--
+--
+function newcontext(cfg)
+   local succ, msg, ctx
+   -- Create the context
+   ctx, msg = context.create(cfg.protocol)
+   if not ctx then return nil, msg end
+   -- Mode
+   succ, msg = context.setmode(ctx, cfg.mode)
+   if not succ then return nil, msg end
+   -- Load the key
+   if cfg.key then
+      succ, msg = context.loadkey(ctx, cfg.key, cfg.password)
+      if not succ then return nil, msg end
+   end
+   -- Load the certificate
+   if cfg.certificate then
+      succ, msg = context.loadcert(ctx, cfg.certificate)
+      if not succ then return nil, msg end
+   end
+   -- Load the CA certificates
+   if cfg.cafile or cfg.capath then
+      succ, msg = context.locations(ctx, cfg.cafile, cfg.capath)
+      if not succ then return nil, msg end
+   end
+   -- Set the verification options
+   succ, msg = optexec(context.setverify, cfg.verify, ctx)
+   if not succ then return nil, msg end
+   -- Set SSL options
+   succ, msg = optexec(context.setoptions, cfg.options, ctx)
+   if not succ then return nil, msg end
+   -- Set the depth for certificate verification
+   if cfg.depth then
+      succ, msg = context.setdepth(ctx, cfg.depth)
+      if not succ then return nil, msg end
+   end
+   return ctx
+end
+
+--
+--
+--
+function wrap(sock, cfg)
+   local ctx, msg
+   if type(cfg) == "table" then
+      ctx, msg = newcontext(cfg)
+      if not ctx then return nil, msg end
+   else
+      ctx = cfg
+   end
+   local s, msg = core.create(ctx)
+   if s then
+      core.setfd(s, sock:getfd())
+      sock:setfd(core.invalidfd)
+      return s
+   end
+   return nil, msg 
+end

lualibs/ssl/https.lua

@@ -0,0 +1,138 @@
+----------------------------------------------------------------------------
+-- LuaSec 0.4.1
+-- Copyright (C) 2009-2011 PUC-Rio
+--
+-- Author: Pablo Musa
+-- Author: Tomas Guisasola
+---------------------------------------------------------------------------
+
+local socket = require("socket")
+local ssl    = require("ssl")
+local ltn12  = require("ltn12")
+local http   = require("socket.http")
+local url    = require("socket.url")
+
+local table  = require("table")
+local string = require("string")
+
+local try          = socket.try
+local type         = type
+local pairs        = pairs
+local getmetatable = getmetatable
+
+module("ssl.https")
+
+_VERSION   = "0.4.1"
+_COPYRIGHT = "LuaSec 0.4.1 - Copyright (C) 2009-2011 PUC-Rio"
+
+-- Default settings
+PORT = 443
+
+local cfg = {
+  protocol = "tlsv1",
+  options  = "all",
+  verify   = "none",
+}
+
+--------------------------------------------------------------------
+-- Auxiliar Functions
+--------------------------------------------------------------------
+
+-- Insert default HTTPS port.
+local function default_https_port(u)
+   return url.build(url.parse(u, {port = PORT}))
+end
+
+-- Convert an URL to a table according to Luasocket needs.
+local function urlstring_totable(url, body, result_table)
+   url = {
+      url = default_https_port(url),
+      method = body and "POST" or "GET",
+      sink = ltn12.sink.table(result_table)
+   }
+   if body then
+      url.source = ltn12.source.string(body)
+      url.headers = {
+         ["content-length"] = #body,
+         ["content-type"] = "application/x-www-form-urlencoded",
+      }
+   end
+   return url
+end
+
+-- Forward calls to the real connection object.
+local function reg(conn)
+   local mt = getmetatable(conn.sock).__index
+   for name, method in pairs(mt) do
+      if type(method) == "function" then
+         conn[name] = function (self, ...)
+                         return method(self.sock, ...)
+                      end
+      end
+   end
+end
+
+-- Return a function which performs the SSL/TLS connection.
+local function tcp(params)
+   params = params or {}
+   -- Default settings
+   for k, v in pairs(cfg) do 
+      params[k] = params[k] or v
+   end
+   -- Force client mode
+   params.mode = "client"
+   -- 'create' function for LuaSocket
+   return function ()
+      local conn = {}
+      conn.sock = try(socket.tcp())
+      local st = getmetatable(conn.sock).__index.settimeout
+      function conn:settimeout(...)
+         return st(self.sock, ...)
+      end
+      -- Replace TCP's connection function
+      function conn:connect(host, port)
+         try(self.sock:connect(host, port))
+         self.sock = try(ssl.wrap(self.sock, params))
+         try(self.sock:dohandshake())
+         reg(self, getmetatable(self.sock))
+         return 1
+      end
+      return conn
+  end
+end
+
+--------------------------------------------------------------------
+-- Main Function
+--------------------------------------------------------------------
+
+-- Make a HTTP request over secure connection.  This function receives
+--  the same parameters of LuaSocket's HTTP module (except 'proxy' and
+--  'redirect') plus LuaSec parameters.
+--
+-- @param url mandatory (string or table)
+-- @param body optional (string)
+-- @return (string if url == string or 1), code, headers, status
+--
+function request(url, body)
+  local result_table = {}
+  local stringrequest = type(url) == "string"
+  if stringrequest then
+    url = urlstring_totable(url, body, result_table)
+  else
+    url.url = default_https_port(url.url)
+  end
+  if http.PROXY or url.proxy then
+    return nil, "proxy not supported"
+  elseif url.redirect then
+    return nil, "redirect not supported"
+  elseif url.create then
+    return nil, "create function not permitted"
+  end
+  -- New 'create' function to establish a secure connection
+  url.create = tcp(url)
+  local res, code, headers, status = http.request(url)
+  if res and stringrequest then
+    return table.concat(result_table), code, headers, status
+  end
+  return res, code, headers, status
+end

lualibs/testwell.lua

@@ -0,0 +1,315 @@
+--
+-- Copyright (C) 2012 Paul Kulchenko
+-- A simple testing library
+-- Based on lua-TestMore : <http://fperrad.github.com/lua-TestMore/>
+-- Copyright (c) 2009-2011 Francois Perrad
+-- This library is licensed under the terms of the MIT/X11 license,
+-- like Lua itself.
+--
+
+local pairs = pairs
+local tostring = tostring
+local type = type
+local _G = _G or _ENV
+
+-----------------------------------------------------------
+
+local tb = {
+  curr_test = 0,
+  good_test = 0,
+  skip_test = 0,
+}
+
+function tb:print(...)
+  print(...)
+end
+
+function tb:note(...)
+  self:print(...)
+end
+
+function tb:diag(...)
+  local arg = {...}
+  for k, v in pairs(arg) do
+    arg[k] = tostring(v)
+  end
+  local msg = table.concat(arg)
+  msg = msg:gsub("\n", "\n# ")
+  msg = msg:gsub("\n# \n", "\n#\n")
+  msg = msg:gsub("\n# $", '')
+  self:print("# " .. msg)
+end
+
+function tb:ok(test, name, more)
+  self.curr_test = self.curr_test + 1
+  self.good_test = self.good_test + (test and 1 or 0)
+  self.skip_test = self.skip_test + (test == nil and 1 or 0)
+  name = tostring(name or '')
+  local out = ''
+  if not test then
+    out = "not "
+  end
+  out = out .. "ok " .. self.curr_test
+  if name ~= '' then
+    out = out .. " - " .. name
+  end
+  self:print(out)
+  if test == false then
+    self:diag("    Failed test " .. (name and ("'" .. name .. "'") or ''))
+    if debug then
+      local info = debug.getinfo(3)
+      local file = info.short_src
+      local line = info.currentline
+      self:diag("    in " .. file .. " at line " .. line .. ".")
+    end
+    self:diag(more)
+  end
+end
+
+function tb:done_testing(reset)
+  local c, g, s = self.curr_test, self.good_test, self.skip_test
+  if reset then
+    self.curr_test = 0
+    self.good_test = 0
+    self.skip_test = 0
+  end
+  return c, g, s
+end
+
+-----------------------------------------------------------
+
+local serpent = (function() ---- include Serpent module for serialization
+local n, v = "serpent", 0.15 -- (C) 2012 Paul Kulchenko; MIT License
+local c, d = "Paul Kulchenko", "Serializer and pretty printer of Lua data types"
+local snum = {[tostring(1/0)]='1/0 --[[math.huge]]',[tostring(-1/0)]='-1/0 --[[-math.huge]]',[tostring(0/0)]='0/0'}
+local badtype = {thread = true, userdata = true}
+local keyword, globals, G = {}, {}, (_G or _ENV)
+for _,k in ipairs({'and', 'break', 'do', 'else', 'elseif', 'end', 'false',
+  'for', 'function', 'goto', 'if', 'in', 'local', 'nil', 'not', 'or', 'repeat',
+  'return', 'then', 'true', 'until', 'while'}) do keyword[k] = true end
+for k,v in pairs(G) do globals[v] = k end -- build func to name mapping
+for _,g in ipairs({'coroutine', 'debug', 'io', 'math', 'string', 'table', 'os'}) do
+  for k,v in pairs(G[g]) do globals[v] = g..'.'..k end end
+
+local function s(t, opts)
+  local name, indent, fatal = opts.name, opts.indent, opts.fatal
+  local sparse, custom, huge = opts.sparse, opts.custom, not opts.nohuge
+  local space, maxl = (opts.compact and '' or ' '), (opts.maxlevel or math.huge)
+  local comm = opts.comment and (tonumber(opts.comment) or math.huge)
+  local seen, sref, syms, symn = {}, {}, {}, 0
+  local function gensym(val) return tostring(val):gsub("[^%w]",""):gsub("(%d%w+)",
+    function(s) if not syms[s] then symn = symn+1; syms[s] = symn end return syms[s] end) end
+  local function safestr(s) return type(s) == "number" and (huge and snum[tostring(s)] or s)
+    or type(s) ~= "string" and tostring(s) -- escape NEWLINE/010 and EOF/026
+    or ("%q"):format(s):gsub("\010","n"):gsub("\026","\\026") end
+  local function comment(s,l) return comm and (l or 0) < comm and ' --[['..tostring(s)..']]' or '' end
+  local function globerr(s,l) return globals[s] and globals[s]..comment(s,l) or not fatal
+    and safestr(tostring(s))..comment('err',l) or error("Can't serialize "..tostring(s)) end
+  local function safename(path, name) -- generates foo.bar, foo[3], or foo['b a r']
+    local n = name == nil and '' or name
+    local plain = type(n) == "string" and n:match("^[%l%u_][%w_]*$") and not keyword[n]
+    local safe = plain and n or '['..safestr(n)..']'
+    return (path or '')..(plain and path and '.' or '')..safe, safe end
+  local alphanumsort = type(opts.sortkeys) == 'function' and opts.sortkeys or function(o, n)
+    local maxn, to = tonumber(n) or 12, {number = 'a', string = 'b'}
+    local function padnum(d) return ("%0"..maxn.."d"):format(d) end
+    table.sort(o, function(a,b)
+      return (o[a] and 0 or to[type(a)] or 'z')..(tostring(a):gsub("%d+",padnum))
+           < (o[b] and 0 or to[type(b)] or 'z')..(tostring(b):gsub("%d+",padnum)) end) end
+  local function val2str(t, name, indent, path, plainindex, level)
+    local ttype, level = type(t), (level or 0)
+    local spath, sname = safename(path, name)
+    local tag = plainindex and
+      ((type(name) == "number") and '' or name..space..'='..space) or
+      (name ~= nil and sname..space..'='..space or '')
+    if seen[t] then
+      table.insert(sref, spath..space..'='..space..seen[t])
+      return tag..'nil'..comment('ref', level)
+    elseif badtype[ttype] then return tag..globerr(t, level)
+    elseif ttype == 'function' then
+      seen[t] = spath
+      local ok, res = pcall(string.dump, t)
+      local func = ok and ((opts.nocode and "function() end" or
+        "loadstring("..safestr(res)..",'@serialized')")..comment(t, level))
+      return tag..(func or globerr(t, level))
+    elseif ttype == "table" then
+      if level >= maxl then return tag..'{}'..comment('max', level) end
+      seen[t] = spath
+      if next(t) == nil then return tag..'{}'..comment(t, level) end -- table empty
+      local maxn, o, out = #t, {}, {}
+      for key = 1, maxn do table.insert(o, key) end
+      for key in pairs(t) do if not o[key] then table.insert(o, key) end end
+      if opts.sortkeys then alphanumsort(o, opts.sortkeys) end
+      for n, key in ipairs(o) do
+        local value, ktype, plainindex = t[key], type(key), n <= maxn and not sparse
+        if opts.ignore and opts.ignore[value] -- skip ignored values; do nothing
+        or sparse and value == nil then -- skipping nils; do nothing
+        elseif ktype == 'table' or ktype == 'function' then
+          if not seen[key] and not globals[key] then
+            table.insert(sref, 'local '..val2str(key,gensym(key),indent)) end
+          table.insert(sref, seen[t]..'['..(seen[key] or globals[key] or gensym(key))
+            ..']'..space..'='..space..(seen[value] or val2str(value,nil,indent)))
+        else
+          if badtype[ktype] then plainindex, key = true, '['..globerr(key, level+1)..']' end
+          table.insert(out,val2str(value,key,indent,spath,plainindex,level+1))
+        end
+      end
+      local prefix = string.rep(indent or '', level)
+      local head = indent and '{\n'..prefix..indent or '{'
+      local body = table.concat(out, ','..(indent and '\n'..prefix..indent or space))
+      local tail = indent and "\n"..prefix..'}' or '}'
+      return (custom and custom(tag,head,body,tail) or tag..head..body..tail)..comment(t, level)
+    else return tag..safestr(t) end -- handle all other types
+  end
+  local sepr = indent and "\n" or ";"..space
+  local body = val2str(t, name, indent) -- this call also populates sref
+  local tail = #sref>0 and table.concat(sref, sepr)..sepr or ''
+  return not name and body or "do local "..body..sepr..tail.."return "..name..sepr.."end"
+end
+
+local function merge(a, b) if b then for k,v in pairs(b) do a[k] = v end end; return a; end
+return { _NAME = n, _COPYRIGHT = c, _DESCRIPTION = d, _VERSION = v, serialize = s,
+  dump = function(a, opts) return s(a, merge({name = '_', compact = true, sparse = true}, opts)) end,
+  line = function(a, opts) return s(a, merge({sortkeys = true, comment = true}, opts)) end,
+  block = function(a, opts) return s(a, merge({indent = '  ', sortkeys = true, comment = true}, opts)) end }
+end)() ---- end of Serpent module
+
+-----------------------------------------------------------
+
+local m = {}
+
+function m.ok(test, name)
+  tb:ok(test, name)
+end
+
+local parms = {comment = false}
+function m.is(got, expected, name)
+  local tgot, texp = type(got), type(expected)
+  local vgot, vexp = serpent.line(got, parms), serpent.line(expected, parms)
+  local pass = vgot == vexp
+  if got == nil then pass = nil end
+  tb:ok(pass, name, not pass and 
+      "         got: " .. vgot .. " (" .. tgot .. ")" ..
+    "\n    expected: " .. vexp .. " (" .. texp .. ")")
+end
+
+function m.isnt(got, expected, name)
+  local tgot, texp = type(got), type(expected)
+  local vgot, vexp = serpent.line(got, parms), serpent.line(expected, parms)
+  local pass = vgot ~= vexp
+  if got == nil then pass = nil end
+  tb:ok(pass, name, not pass and
+      "         got: " .. vgot .. " (" .. tgot .. ")" ..
+    "\n    expected: anything else")
+end
+
+function m.like(got, pattern, name)
+  if type(pattern) ~= 'string' then
+    return tb:ok(false, name, "pattern isn't a string : " .. tostring(pattern))
+  end
+
+  local pass = tostring(got):match(pattern)
+  if got == nil then pass = nil end
+  tb:ok(pass, name, not pass and 
+      "                  '" .. tostring(got) .. "'" ..
+    "\n    doesn't match '" .. pattern .. "'")
+end
+
+function m.unlike(got, pattern, name)
+  if type(pattern) ~= 'string' then
+    return tb:ok(false, name, "pattern isn't a string : " .. tostring(pattern))
+  end
+
+  local pass = not tostring(got):match(pattern)
+  if got == nil then pass = nil end
+  tb:ok(pass, name, not pass and 
+      "                  '" .. tostring(got) .. "'" ..
+    "\n          matches '" .. pattern .. "'")
+end
+
+local cmp = {
+    ['<']  = function (a, b) return a <  b end,
+    ['<='] = function (a, b) return a <= b end,
+    ['>']  = function (a, b) return a >  b end,
+    ['>='] = function (a, b) return a >= b end,
+    ['=='] = function (a, b) return a == b end,
+    ['~='] = function (a, b) return a ~= b end,
+}
+
+function m.cmp_ok(this, op, that, name)
+  local f = cmp[op]
+  if not f then
+    return tb:ok(false, name, "unknown operator : " .. tostring(op))
+  end
+
+  local pass = f(this, that)
+  if this == nil then pass = nil end
+  tb:ok(pass, name, not pass and 
+      "    " .. tostring(this) ..
+    "\n        " .. op ..
+    "\n    " .. tostring(that))
+end
+
+function m.type_ok(val, t, name)
+  if type(t) ~= 'string' then
+    return tb:ok(false, name, "type isn't a string : " .. tostring(t))
+  end
+
+  if type(val) == t then
+    tb:ok(true, name)
+  else
+    tb:ok(false, name,
+      "    " .. tostring(val) .. " isn't a '" .. t .."', it's a '" .. type(val) .. "'")
+  end
+end
+
+function m.diag(...)
+  tb:diag(...)
+end
+
+function m.report()
+  local total, good, skipped = tb:done_testing(true)
+  if total == 0 then return end
+  local failed = total - good - skipped
+  local sum = ("(%d/%d/%d)."):format(good, skipped, total)
+  local num, msg = 0, ""
+  if good > 0 then
+    num, msg = good, msg .. "passed " .. good
+  end
+  if failed > 0 then
+    num, msg = failed, msg .. (#msg > 0 and (skipped > 0 and ", " or " and ") or "")
+      .. "failed " .. failed
+  end
+  if skipped > 0 then 
+    num, msg = skipped, msg .. (#msg > 0 and ((good > 0 and failed > 0 and ',' or '') .." and ") or "")
+      .. "skipped " .. skipped
+  end
+  msg = ("Looks like you %s test%s of %d %s"):format(msg, (num > 1 and 's' or ''), total, sum)
+  if skipped == total then msg = "Looks like you skipped all tests " .. sum end
+  if good == total then msg = "All tests passed " .. sum end
+  tb:note(("1..%d # %s"):format(total, msg))
+end
+
+function m.ismain()
+  for l = 3, 64 do -- only check up to 64 level; no more needed
+    local info = debug.getinfo(l)
+    if not info then return true end
+    if info.func == require then return false end
+  end
+  return true
+end
+
+-- this is needed to call report() when the test object is destroyed
+if _VERSION >= "Lua 5.2" then
+  setmetatable(m, {__gc = m.report})
+else
+  -- keep sentinel alive until 'm' is garbage collected
+  m.sentinel = newproxy(true)
+  getmetatable(m.sentinel).__gc = m.report
+end
+
+for k, v in pairs(m) do -- injection
+  _G[k] = v
+end
+
+return m

spec/glsl.lua

@@ -46,7 +46,7 @@ return {
   },
 
   keywords = {
-    [[int uint half float bool double
+    [[int uint half float bool double atomic_uint binding offset
     vec2 vec3 vec4 dvec2 dvec3 dvec4
     ivec2 ivec3 ivec4 uvec2 uvec3 uvec4 bvec2 bvec3 bvec4
     mat2 mat3 mat4 mat2x2 mat3x3 mat4x4 mat2x3 mat3x2 mat4x2 mat2x4 mat4x3 mat3x4
@@ -63,8 +63,8 @@ return {
     triangles quads equal_spacing isolines fractional_even_spacing lines points
     fractional_odd_spacing cw ccw point_mode lines_adjacency triangles_adjacency
     invocations
-    origin_upper_left pixel_center_integer
-    smooth flat noperspective highp mediump lowp shared packed std140 row_major column_major
+    origin_upper_left pixel_center_integer depth_greater depth_greater depth_greater depth_unchanged
+    smooth flat noperspective highp mediump lowp shared packed std140 std430 row_major column_major buffer
     gl_FrontColor gl_BackColor gl_FrontSecondaryColor gl_BackSecondaryColor gl_Color gl_SecondaryColor
     subroutine gl_Position gl_FragCoord
     gl_VertexID gl_InstanceID gl_Normal gl_Vertex gl_MultiTexCoord0 gl_MultiTexCoord1
@@ -76,22 +76,26 @@ return {
     gl_InvocationID gl_PrimitiveIDIn gl_Layer gl_ViewportIndex gl_FrontFacing
     gl_PointCoord gl_SampleID gl_SamplePosition gl_FragColor
     gl_FragData gl_FragDepth gl_SampleMask
+    gl_NumWorkGroups gl_WorkGroupSize gl_WorkGroupID gl_LocalInvocationID gl_GlobalInvocationID gl_LocalInvocationIndex
+    local_size_x local_size_y local_size_z
 
-    coherent volatile restrict
+    coherent volatile restrict readonly writeonly
     image1D image2D image3D image2DRect imageCube imageBuffer image1DArray image2DArray imageCubeArray image2DMS image2DMSArray
     uimage1D uimage2D uimage3D uimage2DRect uimageCube uimageBuffer uimage1DArray uimage2DArray uimageCubeArray uimage2DMS uimage2DMSArray
     iimage1D iimage2D iimage3D iimage2DRect iimageCube iimageBuffer iimage1DArray iimage2DArray iimageCubeArray iimage2DMS iimage2DMSArray
-    size1x8 size1x16 size1x32 size2x32 size4x32
+    size1x8 size1x16 size1x32 size2x32 size4x32 rgba32f rgba16f rg32f rg16f r32f r16f rgba8 rgba16 r11f_g11f_b10f rgb10_a2ui
+    rgb10_a2i rg16 rg8 r16 r8 rgba32i rgba16i rgba8i rg32i rg16i rg8i r32i r16i r8i rgba32ui rgba16ui rgba8ui rg32ui rg16ui rg8ui
+    r32ui r16ui r8ui rgba16_snorm rgba8_snorm rg16_snorm rg8_snorm r16_snorm r8_snorm
     ]],
 
     [[discard
     radians degrees sin cos tan asin acos atan sinh cosh tanh asinh acosh atanh
     pow exp log exp2 log2 sqrt inversesqrt abs sign floor trunc round
     roundEven ceil fract mod modf min max mix step isnan isinf clamp smoothstep
-    floatBitsToInt intBitsToFloat uintBitsToFloat fma frexp ldexp
+    floatBitsToInt floatBitsToUint intBitsToFloat uintBitsToFloat fma frexp ldexp
     packUnorm2x16 packUnorm4x8 packSnorm4x8
     unpackUnorm2x16 unpackUnorm4x8 unpackSnorm4x8
-    packDouble2x32 unpackDouble2x32
+    packDouble2x32 unpackDouble2x32 packHalf2x16 unpackHalf2x16
     length distance dot cross normalize ftransform faceforward
     reflect refract
     matrixCompMult outerProduct transpose determinant inverse
@@ -138,7 +142,12 @@ return {
     imageAtomicAdd imageAtomicMin imageAtomicMax
     imageAtomicIncWrap imageAtomicDecWrap imageAtomicAnd
     imageAtomicOr imageAtomixXor imageAtomicExchange
-    imageCompSwap memoryBarrier
+    imageCompSwap
+    
+    memoryBarrier groupMemoryBarrier memoryBarrierAtomicCounter memoryBarrierShared memoryBarrierBuffer memoryBarrierImage
+    
+    atomicCounterIncrement atomicCounterDecrement atomicCounter
+    atomicMin atomicMax atomicAdd atomicAnd atomicOr atomicXor atomicExchange atomicCompSwap
 
     x y z w
     xxxx xxxy xxxz xxxw xxyx xxyy xxyz xxyw xxzx xxzy

spec/lua.lua

@@ -1,30 +1,27 @@
 -- authors: Luxinia Dev (Eike Decker & Christoph Kubisch)
 ---------------------------------------------------------
 
+local funcdef = "([A-Za-z_][A-Za-z0-9_%.%:]*)%s*"
+local funccall = "([A-Za-z_][A-Za-z0-9_]*)%s*"
 return {
-  exts = {"lua"},
+  exts = {"lua", "rockspec"},
   lexer = wxstc.wxSTC_LEX_LUA,
   apitype = "lua",
   linecomment = "--",
   sep = "%.:",
   isfncall = function(str)
-    return string.find(str,"([A-Za-z0-9_]+)%s*%(")
+    return string.find(str, funccall .. "%(")
   end,
   isfndef = function(str)
     local l
-    local s,e,cap,par = string.find(str,"function%s+([A-Za-z0-9_]+%s-[%.%:]?%s-[A-Za-z0-9_]*)%s*(%(.-%))")
+    local s,e,cap,par = string.find(str, "function%s+" .. funcdef .. "(%(.-%))")
     -- try to match without brackets now, but only at the beginning of the line
     if (not s) then
-      s,e,cap = string.find(str,"^%s*function%s+([A-Za-z0-9_]+%s-[%.%:]?%s-[A-Za-z0-9_]*)%s*")
+      s,e,cap = string.find(str, "^%s*function%s+" .. funcdef)
     end
     -- try to match "foo = function()"
     if (not s) then
-      s,e,cap,cap1,cap2,par = string.find(str,"(([A-Za-z0-9_]+%s-[%.%:]?%s-)([A-Za-z0-9_]*))%s*=%s*function%s*(%(.-%))")
-      -- check if we captured 'local foo =' instead of 'foo.bar ='
-      if cap1 and cap2 and string.len(cap2) > 0 and not string.find(cap1,'[%.%:]') then
-        cap = cap2
-        s = s + string.len(cap1)
-      end
+      s,e,cap,par = string.find(str, funcdef .. "=%s*function%s*(%(.-%))")
     end
     if (s) then
       l = string.find(string.sub(str,1,s-1),"local%s+$")
@@ -112,13 +109,6 @@ return {
       line = line+1
     end
 
-    if (added) then
-      DisplayOutput("\nTYPES\n")
-      for i,v in pairs(assigns) do
-        DisplayOutput(i,v,"\n")
-      end
-    end
-
     return assigns
   end,
 

spec/ptx.lua

@@ -0,0 +1,437 @@
+-- author: Christoph Kubisch
+---------------------------------------------------------
+
+return {
+  exts = {"ptx",},
+  lexer = wxstc.wxSTC_LEX_CPP,
+  apitype = "ptx",
+  sep = "%.",
+  linecomment = "//",
+
+  isfndef = function(str)
+    local l
+    local s,e,cap = string.find(str,"^%s*([A-Za-z0-9_]+%s+[A-Za-z0-9_]+%s*%(.+%))")
+    if (not s) then
+      s,e,cap = string.find(str,"^%s*([A-Za-z0-9_]+%s+[A-Za-z0-9_]+)%s*%(")
+    end
+    if (cap and (string.find(cap,"^return") or string.find(cap,"else"))) then return end
+    return s,e,cap,l
+  end,
+
+  lexerstyleconvert = {
+    text = {wxstc.wxSTC_C_IDENTIFIER,
+      wxstc.wxSTC_C_VERBATIM,
+      wxstc.wxSTC_C_REGEX,
+      wxstc.wxSTC_C_REGEX,
+      wxstc.wxSTC_C_GLOBALCLASS,},
+
+    lexerdef = {wxstc.wxSTC_C_DEFAULT,},
+    comment = {wxstc.wxSTC_C_COMMENT,
+      wxstc.wxSTC_C_COMMENTLINE,
+      wxstc.wxSTC_C_COMMENTDOC,
+      wxstc.wxSTC_C_COMMENTLINEDOC,
+      wxstc.wxSTC_C_COMMENTDOCKEYWORD,
+      wxstc.wxSTC_C_COMMENTDOCKEYWORDERROR,},
+    stringtxt = {wxstc.wxSTC_C_STRING,
+      wxstc.wxSTC_C_CHARACTER,
+      wxstc.wxSTC_C_UUID,},
+    stringeol = {wxstc.wxSTC_C_STRINGEOL,},
+    preprocessor= {wxstc.wxSTC_C_PREPROCESSOR,},
+    operator = {wxstc.wxSTC_C_OPERATOR,},
+    number = {wxstc.wxSTC_C_NUMBER,
+      wxstc.wxSTC_C_WORD},
+
+    keywords0 = {wxstc.wxSTC_C_WORD,},
+    keywords1 = {wxstc.wxSTC_C_WORD2,},
+  },
+
+  keywords = {
+[[
+version
+target
+address_size
+
+entry
+func
+
+branchtargets
+calltargets
+callprototype
+
+maxnreg
+maxntid
+reqntid
+minnctapersm
+maxnctapersm
+pragma
+
+section
+file
+loc
+
+extern
+visible
+
+pragma
+
+align
+file
+maxntid
+shared
+branchtargets
+func
+minnctapersm
+sreg
+callprototype
+global
+param
+target
+calltargets
+local
+pragma
+tex
+const
+loc
+reg
+version
+entry
+maxnctapersm
+reqntid
+visible
+extern
+maxnreg
+section
+
+s8
+s16
+s32
+s64
+u8
+u16
+u32
+u64
+f16
+f32
+f64
+b8
+b16
+b32
+b64
+pred
+
+rn
+rz
+rm
+rp
+
+rni
+rzi
+rmi
+rpi
+
+ca 
+cg
+cs 
+lu
+cv
+
+wb
+cg
+cs
+wt
+
+texref
+samplerref
+surfref
+
+sat
+ftz
+
+cc
+
+hi
+lo
+wide
+
+f4e
+b4e
+rc8
+ecl
+ecr
+rc16
+
+finite
+infinite
+number
+notanumber
+normal
+subnormal
+
+approx
+full
+
+eq
+ne
+lt
+le
+gt
+ge
+
+equ
+neu
+ltu
+leu
+gtu
+geu
+
+num
+nan
+
+ls
+hs
+
+volatile
+
+v2
+v4
+
+L1
+L2
+
+1d
+2d
+3d
+a1d
+a2d
+
+width
+height
+depth
+channel_data_type
+channel_order
+normalized_coords
+
+force_unnormalized_coords
+filter_mode
+addr_mode_0
+addr_mode_1
+addr_mode_2
+
+trap
+clamp
+zero
+
+all
+any
+uni
+ballot
+
+sync
+arrive
+red
+
+cta
+gl
+sys
+
+and
+or
+xor
+cas
+exch
+add
+inc
+dec
+min
+max
+
+b0
+b1
+b2
+b3
+h0
+h1
+wrap
+shr7
+shr15
+
+byte 
+4byte 
+quad 
+4byte 
+quad
+
+b8 
+b32 
+b64 
+b32 
+b64
+]],
+
+-- functions
+
+[[
+add
+sub
+add.cc
+addc
+sub.cc
+subc
+mul
+mad
+mul24
+mad24
+sad
+div
+rem
+abs
+neg
+min
+max
+popc
+clz
+bfind
+brev
+bfe
+bfi
+prmt
+
+rcp
+sqrt
+rsqrt
+sin
+cos
+lg2
+ex2
+fma
+
+set
+setp
+selp
+slct
+
+and
+or
+xor
+not
+cnot
+shl
+shr
+
+mov
+ld
+ldu
+st
+prefetch
+prefetchu
+isspacep
+cvta
+cvt
+
+tex
+tld4
+txq
+suld
+sust
+sured
+suq
+
+bra
+call
+ret
+exit
+
+bar
+membar
+atom
+red
+vote
+
+vadd
+vsub
+vabsdiff
+vmin
+vmax
+vshl
+vshr
+vmad
+vset
+
+trap
+brkpt
+pmevent
+
+%clock
+%laneid
+%lanemask_gt
+%pm0
+%pm1
+%pm2
+%pm3
+%clock64
+%lanemask_eq
+%nctaid
+%smid
+%ctaid
+%lanemask_le
+%ntid
+%tid
+%envreg0
+%envreg1
+%envreg2
+%envreg3
+%envreg4
+%envreg5
+%envreg6
+%envreg7
+%envreg8
+%envreg9
+%envreg10
+%envreg11
+%envreg12
+%envreg13
+%envreg14
+%envreg15
+%envreg16
+%envreg17
+%envreg18
+%envreg19
+%envreg20
+%envreg21
+%envreg22
+%envreg23
+%envreg24
+%envreg25
+%envreg26
+%envreg27
+%envreg28
+%envreg29
+%envreg30
+%envreg31
+%lanemask_lt
+%nsmid
+%warpid
+%gridid
+%lanemask_ge
+%nwarpid
+WARP_SZ
+nearest
+linear
+wrap
+mirror
+clamp_ogl
+clamp_to_edge
+clamp_to_border
+
+sm_20
+sm_10
+sm_11
+sm_12
+sm_13
+texmode_unified
+texmode_independent
+map_f64_to_f32
+]],
+
+  },
+}

src/defs.lua

@@ -13,7 +13,7 @@
 
 -- style definition
 -- ----------------------------------------------------
--- all entries are optiona
+-- all entries are optional
 stattr = {
   fg = {r,g,b}, -- foreground color 0-255
   bg = {r,g,b}, -- background color
@@ -21,6 +21,13 @@ stattr = {
   b = false, -- bold
   u = false, -- underline
   fill = true, -- fill to lineend
+  -- fn = "Lucida Console", -- font Face Name
+  -- fx = 11, -- font size
+  -- hs = true or {r,g,b}, -- turn hotspot on
+  --   use the specified color as activeForeground
+  --   use "hs = true", to turn it on without changing the color
+  --   HotspotActiveUnderline and HotspotSingleLine are on automatically
+  -- v = true, -- visibility for symbols of the current style
 }
 
 style = {
@@ -105,6 +112,12 @@ config = {
     whitespace = false,
     autotabs = true, -- if true test for tabs after file load,
     -- sets "usetabs" to true for this file
+    calltipdelay = nil, -- delay to show calltip (in ms)
+  },
+
+  default = {
+    name = 'untitled',
+    fullname = 'untitled.lua',
   },
 
   debugger = {
@@ -159,6 +172,10 @@ config = {
 
   singleinstanceport = 0xe493,
   -- UDP port for single instance communication
+
+  activateoutput = false, -- activate output/console on Run/Debug/Compile
+  unhidewindow = false, -- to unhide a gui window
+  allowinteractivescript = false, -- allow interaction in the output window
 }
 
 -- application engine
@@ -290,21 +307,18 @@ debuginterface = {
   breakpoint = function(self,file,line,state) end, -- set breakpoint state
 
   -- returns result table if successful
-  evaluate = function(self, expressions, fnSetValues) end, -- for watches tables expected
-
-  -- NYI getstack = function(self, fnSetValues ) end, -- get stack information
+  evaluate = function(self, expressions, fnSetValues) end, -- for watches tables
+  stack = function(self) end, -- get stack information
 }
 
 -- interpreter definition-- ----------------------------------------------------
 interpreter = {
   name = "",
   description = "",
-  api = {"apifile_without_extension"} -- optional to limit loaded lua apis
-  frun = function(self,wfilename,withdebugger)
-  end,
-  fprojdir = function(self,wfilename)
-    return "projpath_from_filename" -- optional
-  end,
-  fattachdebug = function(self) end, -- optional
+  api = {"apifile_without_extension"} -- (opt) to limit loaded lua apis
+  frun = function(self,wfilename,withdebugger) end,
+  fprojdir = function(self,wfilename) return "projpath_from_filename" end, -- (opt)
+  fattachdebug = function(self) end, -- (opt)
   hasdebugger = false, -- if debugging is available
+  scratchextloop = false, -- (opt) if scratchpad requires handling for external loop
 }

src/editor/autocomplete.lua

@@ -89,8 +89,8 @@ local function addAPI(apifile,only,subapis,known) -- relative to API directory
 end
 
 local function loadallAPIs (only,subapis,known)
-  for i,dir in ipairs(FileSysGet(".\\api\\*.*",wx.wxDIR)) do
-    local files = FileSysGet(dir.."\\*.*",wx.wxFILE)
+  for i,dir in ipairs(FileSysGet("./api/*",wx.wxDIR)) do
+    local files = FileSysGet(dir.."/*.*",wx.wxFILE)
     for i,file in ipairs(files) do
       if file:match "%.lua$" then
         addAPI(file,only,subapis,known)
@@ -239,7 +239,7 @@ end
 
 function GetTipInfo(editor, content, short)
   local caller = content:match("([%w_]+)%(%s*$")
-  local class = caller and content:match("([%w_%.]+)[%.:]"..caller.."%(%s*$")
+  local class = caller and content:match("([%w_]+)[%.:]"..caller.."%(%s*$")
   local tip = editor.api.tip
 
   local classtab = short and tip.shortfinfoclass or tip.finfoclass

src/editor/commands.lua

@@ -5,9 +5,10 @@ local ide = ide
 local frame = ide.frame
 local notebook = frame.notebook
 local openDocuments = ide.openDocuments
+local uimgr = frame.uimgr
 
 function NewFile(event)
-  local editor = CreateEditor("untitled.lua")
+  local editor = CreateEditor(ide.config.default.fullname)
   SetupKeywords(editor, "lua")
 end
 
@@ -26,39 +27,33 @@ local function findDocumentToReuse()
 end
 
 function LoadFile(filePath, editor, file_must_exist)
-  filePath = wx.wxFileName(filePath):GetFullPath()
-  local cmpName = string.lower(string.gsub(filePath, "\\", "/"))
-  
   -- prevent files from being reopened again
   if (not editor) then
+    local filePath = wx.wxFileName(filePath)
     for id, doc in pairs(openDocuments) do
-      local docName = doc.filePath and string.lower(string.gsub(doc.filePath, "\\", "/"))
-      if cmpName == docName then
+      if doc.filePath and filePath:SameAs(wx.wxFileName(doc.filePath)) then
         notebook:SetSelection(doc.index)
         return doc.editor
       end
     end
   end
+  filePath = wx.wxFileName(filePath):GetFullPath()
 
   -- if not opened yet, try open now
-  local file_text = ""
-  local handle = io.open(filePath, "rb")
-  if handle then
-    file_text = handle:read("*a")
+  local file_text = FileRead(filePath)
+  if file_text then
     if GetConfigIOFilter("input") then
       file_text = GetConfigIOFilter("input")(filePath,file_text)
     end
-    handle:close()
   elseif file_must_exist then
     return nil
   end
 
-  if not editor then
-    editor = findDocumentToReuse()
-  end
-  if not editor then
-    editor = CreateEditor(wx.wxFileName(filePath):GetFullName() or "untitled.lua")
-  end
+  local current = editor and editor:GetCurrentPos()
+  editor = editor
+    or findDocumentToReuse()
+    or CreateEditor(wx.wxFileName(filePath):GetFullName()
+      or ide.config.default.fullname)
 
   editor:Clear()
   editor:ClearAll()
@@ -66,6 +61,7 @@ function LoadFile(filePath, editor, file_must_exist)
   editor:MarkerDeleteAll(BREAKPOINT_MARKER)
   editor:MarkerDeleteAll(CURRENT_LINE_MARKER)
   editor:AppendText(file_text)
+  if current then editor:GotoPos(current) end
   if (ide.config.editor.autotabs) then
     local found = string.find(file_text,"\t") ~= nil
     editor:SetUseTabs(found)
@@ -82,8 +78,10 @@ function LoadFile(filePath, editor, file_must_exist)
   IndicateFunctions(editor)
 
   SettingsAppendFileToHistory(filePath)
-  
-  SetEditorSelection(nil)
+
+  -- activate the editor; this is needed for those cases when the editor is
+  -- created from some other element, for example, from a project tree.
+  SetEditorSelection()
 
   return editor
 end
@@ -113,11 +111,11 @@ function OpenFile(event)
     "",
     "",
     exts,
-    wx.wxOPEN + wx.wxFILE_MUST_EXIST)
+    wx.wxFD_OPEN + wx.wxFD_FILE_MUST_EXIST)
   if fileDialog:ShowModal() == wx.wxID_OK then
     if not LoadFile(fileDialog:GetPath(), nil, true) then
       wx.wxMessageBox("Unable to load file '"..fileDialog:GetPath().."'.",
-        "wxLua Error",
+        "Error",
         wx.wxOK + wx.wxCENTRE, ide.frame)
     end
   end
@@ -129,24 +127,15 @@ function SaveFile(editor, filePath)
   if not filePath then
     return SaveFileAs(editor)
   else
-    filePath = filePath:gsub("\\","/")
+    if (ide.config.savebak) then FileRename(filePath, filePath..".bak") end
 
-    if (ide.config.savebak) then
-      local backPath = filePath..".bak"
-      os.remove(backPath)
-      os.rename(filePath, backPath)
+    local st = editor:GetText()
+    if GetConfigIOFilter("output") then
+      st = GetConfigIOFilter("output")(filePath,st)
     end
 
-    local handle = io.open(filePath, "wb")
-    if handle then
-      local st = editor:GetText()
-
-      if GetConfigIOFilter("output") then
-        st = GetConfigIOFilter("output")(filePath,st)
-      end
-      handle:write(st)
-      handle:close()
-      --editor:EmptyUndoBuffer()
+    local ok, err = FileWrite(filePath, st)
+    if ok then
       editor:SetSavePoint()
       local id = editor:GetId()
       openDocuments[id].filePath = filePath
@@ -155,9 +144,9 @@ function SaveFile(editor, filePath)
       SetDocumentModified(id, false)
       return true
     else
-      wx.wxMessageBox("Unable to save file '"..filePath.."'.",
-        "wxLua Error Saving",
-        wx.wxOK + wx.wxCENTRE, ide.frame)
+      wx.wxMessageBox("Unable to save file '"..filePath.."': "..err,
+        "Error",
+        wx.wxICON_ERROR + wx.wxOK + wx.wxCENTRE, ide.frame)
     end
   end
 
@@ -170,7 +159,7 @@ function SaveFileAs(editor)
   local filePath = openDocuments[id].filePath
   if (not filePath) then
     filePath = FileTreeGetDir()
-    filePath = (filePath or "").."untitled"
+    filePath = (filePath or "")..ide.config.default.name
   end
 
   local fn = wx.wxFileName(filePath)
@@ -182,14 +171,18 @@ function SaveFileAs(editor)
     fn:GetPath(wx.wxPATH_GET_VOLUME),
     fn:GetFullName(),
     exts,
-    wx.wxSAVE)
+    wx.wxFD_SAVE)
 
   if fileDialog:ShowModal() == wx.wxID_OK then
     local filePath = fileDialog:GetPath()
 
     if SaveFile(editor, filePath) then
+      SetEditorSelection() -- update title of the editor
+      FileTreeRefresh() -- refresh the tree to reflect the new file
+      FileTreeMarkSelected(filePath)
       SetupKeywords(editor, GetFileExt(filePath))
       IndicateFunctions(editor)
+      if MarkupStyle then MarkupStyle(editor) end
       saved = true
     end
   end
@@ -218,7 +211,10 @@ local function removePage(index)
   selectIndex = selectIndex ~= index and selectIndex
 
   local delid = nil
-  for id, document in pairs(openDocuments) do
+  for id, document in pairsSorted(openDocuments,
+    function(a, b) -- sort by document index
+      return openDocuments[a].index < openDocuments[b].index
+    end) do
     local wasselected = document.index == selectIndex
     if document.index < index then
       prevIndex = document.index
@@ -250,7 +246,7 @@ local function removePage(index)
     notebook:SetSelection(prevIndex)
   end
 
-  SetEditorSelection(nil) -- will use notebook GetSelection to update
+  SetEditorSelection() -- will use notebook GetSelection to update
 end
 
 function ClosePage(selection)
@@ -258,6 +254,19 @@ function ClosePage(selection)
   local id = editor:GetId()
   if SaveModifiedDialog(editor, true) ~= wx.wxID_CANCEL then
     DynamicWordsRemoveAll(editor)
+    local debugger = ide.debugger
+    -- check if the window with the scratchpad running is being closed
+    if debugger and debugger.scratchpad and debugger.scratchpad.editor == editor then
+      DebuggerScratchpadOff()
+    end
+    -- check if the debugger is running and is using the current window
+    -- abort the debugger if the current marker is in the window being closed
+    -- also abort the debugger if it is running, as we don't know what
+    -- window will need to be activated when the debugger is paused
+    if debugger and debugger.pid and
+      (debugger.running or editor:MarkerNext(0, CURRENT_LINE_MARKER_VALUE) >= 0) then
+      debugger.terminate()
+    end
     removePage(ide.openDocuments[id].index)
   end
 end
@@ -271,12 +280,8 @@ function SaveModifiedDialog(editor, allow_cancel)
   local filePath = document.filePath
   local fileName = document.fileName
   if document.isModified then
-    local message
-    if fileName then
-      message = "Save changes to '"..fileName.."' before exiting?"
-    else
-      message = "Save changes to 'untitled' before exiting?"
-    end
+    local message = "Do you want to save the changes to '"
+      ..(fileName or ide.config.default.name).."'?"
     local dlg_styles = wx.wxYES_NO + wx.wxCENTRE + wx.wxICON_QUESTION
     if allow_cancel then dlg_styles = dlg_styles + wx.wxCANCEL end
     local dialog = wx.wxMessageDialog(ide.frame, message,
@@ -299,8 +304,13 @@ function SaveOnExit(allow_cancel)
     if (SaveModifiedDialog(document.editor, allow_cancel) == wx.wxID_CANCEL) then
       return false
     end
+  end
 
-    document.isModified = false
+  -- if all documents have been saved or refused to save, then mark those that
+  -- are still modified as not modified (they don't need to be saved)
+  -- to keep their tab names correct
+  for id, document in pairs(openDocuments) do
+    if document.isModified then SetDocumentModified(id, false) end
   end
 
   return true
@@ -385,20 +395,23 @@ function ClearAllCurrentLineMarkers()
   end
 end
 
-function CompileProgram(editor)
-  local editorText = editor:GetText()
+function CompileProgram(editor, quiet)
+  -- remove shebang line (#!) as it throws a compilation error as
+  -- loadstring() doesn't allow it even though lua/loadfile accepts it.
+  -- replace with a new line to keep the number of lines the same.
+  local editorText = editor:GetText():gsub("^#!.-\n", "\n")
   local id = editor:GetId()
   local filePath = DebuggerMakeFileName(editor, openDocuments[id].filePath)
-  local ret, errMsg, line_num = wxlua.CompileLuaScript(editorText, filePath)
-  if ide.frame.menuBar:IsChecked(ID_CLEAROUTPUT) then
-    ClearOutput()
-  end
+  local _, errMsg, line_num = wxlua.CompileLuaScript(editorText, filePath)
+
+  if ide.frame.menuBar:IsChecked(ID_CLEAROUTPUT) then ClearOutput() end
 
   if line_num > -1 then
-    DisplayOutput("Compilation error on line number :"..tostring(line_num).."\n"..errMsg.."\n")
-    editor:GotoLine(line_num-1)
+    DisplayOutput("Compilation error on line "..tostring(line_num)..":\n"..
+      errMsg:gsub("Lua:.-\n", "").."\n")
+    if not quiet then editor:GotoLine(line_num-1) end
   else
-    DisplayOutput("Compilation successful.\n")
+    if not quiet then DisplayOutput("Compilation successful.\n") end
   end
 
   return line_num == -1 -- return true if it compiled ok
@@ -470,23 +483,47 @@ function SetOpenFiles(nametab,index)
   notebook:SetSelection(index or 0)
 end
 
+local beforeFullScreenPerspective
+function ShowFullScreen(setFullScreen)
+  if setFullScreen then
+    beforeFullScreenPerspective = uimgr:SavePerspective()
+    uimgr:GetPane("bottomnotebook"):Show(false)
+    uimgr:GetPane("projpanel"):Show(false)
+    SetEditorSelection() -- make sure the focus is on the editor
+  elseif beforeFullScreenPerspective then
+    uimgr:LoadPerspective(beforeFullScreenPerspective)
+    beforeFullScreenPerspective = nil
+  end
+
+  uimgr:GetPane("toolBar"):Show(not setFullScreen)
+  uimgr:Update()
+  -- protect from systems that don't have ShowFullScreen (GTK on linux?)
+  pcall(function() frame:ShowFullScreen(setFullScreen) end)
+end
+
 function CloseWindow(event)
-  exitingProgram = true -- don't handle focus events
+  ide.exitingProgram = true -- don't handle focus events
 
   if not SaveOnExit(event:CanVeto()) then
     event:Veto()
-    exitingProgram = false
+    ide.exitingProgram = false
     return
   end
 
+  ShowFullScreen(false)
   SettingsSaveProjectSession(FileTreeGetProjects())
   SettingsSaveFileSession(GetOpenFiles())
   SettingsSaveView()
   SettingsSaveFramePosition(ide.frame, "MainFrame")
   SettingsSaveEditorSettings()
-  DebuggerCloseWatchWindow()
-  DebuggerKillClient()
+  if DebuggerCloseWatchWindow then DebuggerCloseWatchWindow() end
+  if DebuggerCloseStackWindow then DebuggerCloseStackWindow() end
+  if DebuggerShutdown then DebuggerShutdown() end
   ide.settings:delete() -- always delete the config
   event:Skip()
+
+  -- without explicit exit() the IDE crashes with SIGILL exception when closed
+  -- on MacOS compiled under 64bit with wxwidgets 2.9.3
+  if ide.osname == "Macintosh" then os.exit() end
 end
 frame:Connect(wx.wxEVT_CLOSE_WINDOW, CloseWindow)

src/editor/editor.lua

@@ -3,11 +3,9 @@
 ---------------------------------------------------------
 local wxkeywords = nil -- a string of the keywords for scintilla of wxLua's wx.XXX items
 
-local in_evt_focus = false -- true when in editor focus event to avoid recursion
 local editorID = 100 -- window id to create editor pages with, incremented for new editors
 
 local openDocuments = ide.openDocuments
-local ignoredFilesList = ide.ignoredFilesList
 local statusBar = ide.frame.statusBar
 local notebook = ide.frame.notebook
 local funclist = ide.frame.toolBar.funclist
@@ -17,10 +15,10 @@ local projcombobox = ide.frame.projpanel.projcombobox
 -- ----------------------------------------------------------------------------
 -- Update the statusbar text of the frame using the given editor.
 -- Only update if the text has changed.
-statusTextTable = { "OVR?", "R/O?", "Cursor Pos" }
+local statusTextTable = { "OVR?", "R/O?", "Cursor Pos" }
 
 -- set funclist font to be the same as the combobox in the project dropdown
-funclist:SetFont(projcombobox:GetFont())
+funclist:SetFont(ide.font.fNormal)
 
 local function updateStatusText(editor)
   local texts = { "", "", "" }
@@ -29,9 +27,10 @@ local function updateStatusText(editor)
     local line = editor:LineFromPosition(pos)
     local col = 1 + pos - editor:PositionFromLine(line)
 
-    texts = { iff(editor:GetOvertype(), "OVR", "INS"),
+    texts = {
+      iff(editor:GetOvertype(), "OVR", "INS"),
       iff(editor:GetReadOnly(), "R/O", "R/W"),
-      "Ln "..tostring(line + 1).." Col "..tostring(col) }
+      "Ln: "..tostring(line + 1).." Col: "..tostring(col) }
   end
 
   if ide.frame then
@@ -62,6 +61,11 @@ local function updateBraceMatch(editor)
   pos = (match[char] and pos) or (charp and match[charp] and posp)
 
   if (pos) then
+    -- don't match brackets in markup comments
+    local style = bit.band(editor:GetStyleAt(pos), 31)
+    if MarkupIsSpecial and MarkupIsSpecial(style)
+      or editor.spec.iscomment[style] then return end
+
     local pos2 = editor:BraceMatch(pos)
     if (pos2 == wxstc.wxSTC_INVALID_POSITION) then
       editor:BraceBadLight(pos)
@@ -102,7 +106,7 @@ local function isFileAlteredOnDisk(editor)
         wx.wxMessageBox(fileName.." is no longer on the disk.",
           GetIDEString("editormessage"),
           wx.wxOK + wx.wxCENTRE, ide.frame)
-      elseif modTime:IsValid() and oldModTime:IsEarlierThan(modTime) then
+      elseif not editor:GetReadOnly() and modTime:IsValid() and oldModTime:IsEarlierThan(modTime) then
         local ret = wx.wxMessageBox(fileName.." has been modified on disk.\nDo you want to reload it?",
           GetIDEString("editormessage"),
           wx.wxYES_NO + wx.wxCENTRE, ide.frame)
@@ -118,11 +122,12 @@ end
 -- ----------------------------------------------------------------------------
 -- Get/Set notebook editor page, use nil for current page, returns nil if none
 function GetEditor(selection)
-  local editor = nil
   if selection == nil then
     selection = notebook:GetSelection()
   end
-  if (selection >= 0) and (selection < notebook:GetPageCount()) and (notebook:GetPage(selection):GetClassInfo():GetClassName()=="wxStyledTextCtrl") then
+  local editor
+  if (selection >= 0) and (selection < notebook:GetPageCount())
+    and (notebook:GetPage(selection):GetClassInfo():GetClassName()=="wxStyledTextCtrl") then
     editor = notebook:GetPage(selection):DynamicCast("wxStyledTextCtrl")
   end
   return editor
@@ -135,19 +140,18 @@ function SetEditorSelection(selection)
   statusBar:SetStatusText("",1)
   ide.frame:SetTitle(getFileTitle(editor))
 
-  FileTreeMarkSelected('')
-
   if editor then
     if funclist:IsEmpty() then funclist:Append('Jump to a function definition...', 0) end
     funclist:SetSelection(0)
 
     editor:SetFocus()
     editor:SetSTCFocus(true)
-    isFileAlteredOnDisk(editor)
     local id = editor:GetId()
     if openDocuments[id] and openDocuments[id].filePath then
       FileTreeMarkSelected(openDocuments[id].filePath)
     end
+  else
+    FileTreeMarkSelected('')
   end
 end
 
@@ -157,7 +161,7 @@ function GetEditorFileAndCurInfo(nochecksave)
     return
   end
 
-  local id = editor:GetId();
+  local id = editor:GetId()
   local filepath = openDocuments[id].filePath
   if (nochecksave and not filepath) then
     return
@@ -178,7 +182,7 @@ end
 
 -- Set if the document is modified and update the notebook page text
 function SetDocumentModified(id, modified)
-  local pageText = openDocuments[id].fileName or "untitled.lua"
+  local pageText = openDocuments[id].fileName or ide.config.default.fullname
 
   if modified then
     pageText = "* "..pageText
@@ -188,6 +192,83 @@ function SetDocumentModified(id, modified)
   notebook:SetPageText(openDocuments[id].index, pageText)
 end
 
+function EditorAutoComplete(editor)
+  if (editor == nil or not editor.spec) then return end
+
+  -- retrieve the current line and get a string to the current cursor position in the line
+  local pos = editor:GetCurrentPos()
+  local line = editor:GetCurrentLine()
+  local linetx = editor:GetLine(line)
+  local linestart = editor:PositionFromLine(line)
+  local localpos = pos-linestart
+
+  local lt = linetx:sub(1,localpos)
+  lt = lt:gsub("%s*("..editor.spec.sep..")%s*",function(a) return a end)
+  lt = lt:gsub("%s*%b[]%s*","")
+  lt = lt:gsub("%s*%b()%s*","")
+  lt = lt:gsub("%s*%b{}%s*","")
+  lt = lt:match("[^%[%(%s]*$")
+  lt = lt:gsub("%s","")
+
+  -- know now which string is to be completed
+  local userList = CreateAutoCompList(editor,lt)
+  if userList and string.len(userList) > 0 then
+    editor:UserListShow(1, userList)
+  elseif editor:AutoCompActive() then
+    editor:AutoCompCancel()
+  end
+end
+
+function EditorCallTip(editor, pos)
+  local line = editor:LineFromPosition(pos)
+  local linetx = editor:GetLine(line)
+  local linestart = editor:PositionFromLine(line)
+  local localpos = pos-linestart
+
+  local ident = "([a-zA-Z_0-9][a-zA-Z_0-9%.%:]*)"
+  local linetxtopos = linetx:sub(1,localpos)
+  linetxtopos = linetxtopos..")"
+  linetxtopos = linetxtopos:match(ident .. "%b()$")
+
+  local tip = linetxtopos and GetTipInfo(editor,linetxtopos.."(",false)
+  if ide.debugger and ide.debugger.server then
+    local selected = editor:GetSelectionStart() ~= editor:GetSelectionEnd()
+      and pos >= editor:GetSelectionStart() and pos <= editor:GetSelectionEnd()
+
+    -- check if we have a selected text or an identifier
+    -- for an identifier, check fragments on the left and on the right.
+    -- this is to match 'io' in 'i^o.print' and 'io.print' in 'io.pr^int'.
+    -- remove square brackets to make tbl[index].x show proper values.
+    local start = linetx:sub(1,localpos)
+      :gsub("%b[]", function(s) return ("."):rep(#s) end)
+      :find(ident.."$")
+
+    -- check if the style is the right one; this is to ignore
+    -- comments, strings, numbers (to avoid '1 = 1'), keywords, and such
+    if start and not selected then
+      local style = bit.band(editor:GetStyleAt(linestart+start),31)
+      if editor.spec.iscomment[style]
+      or editor.spec.isstring[style]
+      or style == wxstc.wxSTC_LUA_NUMBER
+      or style == wxstc.wxSTC_LUA_WORD then
+        return -- don't do anything for strings or comments or numbers
+      end
+    end
+
+    local right = linetx:sub(localpos+1,#linetx):match("^[a-zA-Z_0-9]*")
+    local var = selected and editor:GetSelectedText()
+      or (start and linetx:sub(start,localpos):gsub(":",".")..right or nil)
+    if var then
+      local limit = 128
+      ide.debugger.quickeval(var, function(val)
+        if #val > limit then val = val:sub(1, limit-3).."..." end
+        editor:CallTipShow(pos, val) end)
+    end
+  elseif tip then
+    editor:CallTipShow(pos, tip)
+  end
+end
+
 -- ----------------------------------------------------------------------------
 -- Create an editor and add it to the notebook
 function CreateEditor(name)
@@ -203,8 +284,8 @@ function CreateEditor(name)
   editor:SetBufferedDraw(true)
   editor:StyleClearAll()
 
-  editor:SetFont(ide.font)
-  editor:StyleSetFont(wxstc.wxSTC_STYLE_DEFAULT, ide.font)
+  editor:SetFont(ide.font.eNormal)
+  editor:StyleSetFont(wxstc.wxSTC_STYLE_DEFAULT, ide.font.eNormal)
 
   editor:SetTabWidth(ide.config.editor.tabwidth or 4)
   editor:SetIndent(ide.config.editor.tabwidth or 4)
@@ -214,10 +295,8 @@ function CreateEditor(name)
 
   if (ide.config.editor.usewrap) then
     editor:SetWrapMode(wxstc.wxSTC_WRAP_WORD)
-    editor:SetWrapStartIndent(2)
+    editor:SetWrapStartIndent(0)
     editor:SetWrapVisualFlagsLocation(wxstc.wxSTC_WRAPVISUALFLAGLOC_END_BY_TEXT)
-    editor:SetWrapVisualFlags(wxstc.wxSTC_WRAPVISUALFLAG_START)
-    editor:WrapCount(80)
   end
 
   editor:SetCaretLineVisible(ide.config.editor.caretline and 1 or 0)
@@ -257,6 +336,10 @@ function CreateEditor(name)
   editor:MarkerDefine(wxstc.wxSTC_MARKNUM_FOLDERMIDTAIL, wxstc.wxSTC_MARK_TCORNER, wx.wxWHITE, grey)
   grey:delete()
 
+  if ide.config.editor.calltipdelay and ide.config.editor.calltipdelay > 0 then
+    editor:SetMouseDwellTime(ide.config.editor.calltipdelay)
+  end
+
   editor:AutoCompSetIgnoreCase(ide.config.acandtip.ignorecase)
   if (ide.config.acandtip.strategy > 0) then
     editor:AutoCompSetAutoHide(0)
@@ -305,7 +388,6 @@ function CreateEditor(name)
         DynamicWordsRem("pre",editor,nil,editor:LineFromPosition(event:GetPosition()), numlines)
       end
       if (bit.band(evtype,wxstc.wxSTC_MOD_BEFOREINSERT) ~= 0) then
-        local pos = event:GetPosition()
         DynamicWordsRem("pre",editor,nil,editor:LineFromPosition(event:GetPosition()), 0)
       end
     end)
@@ -321,7 +403,7 @@ function CreateEditor(name)
       local linestart = editor:PositionFromLine(line)
       local localpos = pos-linestart
 
-      linetxtopos = linetx:sub(1,localpos)
+      local linetxtopos = linetx:sub(1,localpos)
 
       if (ch == char_CR and eol==2) or (ch == char_LF and eol==0) then
         if (line > 0) then
@@ -334,26 +416,37 @@ function CreateEditor(name)
             editor:GotoPos(pos+indent)
           end
         end
-      elseif ch == ("("):byte() then
 
+      elseif ch == ("("):byte() then
         local tip = GetTipInfo(editor,linetxtopos,ide.config.acandtip.shorttip)
         if tip then
           editor:CallTipShow(pos,tip)
         end
 
       elseif ide.config.autocomplete then -- code completion prompt
-
         local trigger = linetxtopos:match("["..editor.spec.sep.."%w_]+$")
-
         if (trigger and (#trigger > 1 or trigger:match("[%.:]"))) then
-          -- defined in menu_edit.lua
-          local commandEvent = wx.wxCommandEvent(wx.wxEVT_COMMAND_MENU_SELECTED,
-            ID_AUTOCOMPLETE)
-          wx.wxPostEvent(ide.frame, commandEvent)
+          ide.frame:AddPendingEvent(wx.wxCommandEvent(
+            wx.wxEVT_COMMAND_MENU_SELECTED, ID_AUTOCOMPLETE))
         end
       end
     end)
 
+  editor:Connect(wxstc.wxEVT_STC_DWELLSTART,
+    function (event)
+      local position = editor:PositionFromPointClose(event:GetX(),event:GetY())
+      if position ~= wxstc.wxSTC_INVALID_POSITION then
+        EditorCallTip(editor, position)
+      end
+      event:Skip()
+    end)
+
+  editor:Connect(wxstc.wxEVT_STC_DWELLEND,
+    function (event)
+      if editor:CallTipActive() then editor:CallTipCancel() end
+      event:Skip()
+    end)
+
   editor:Connect(wxstc.wxEVT_STC_USERLISTSELECTION,
     function (event)
       local pos = editor:GetCurrentPos()
@@ -363,42 +456,73 @@ function CreateEditor(name)
     end)
 
   editor:Connect(wxstc.wxEVT_STC_SAVEPOINTREACHED,
-    function (event)
+    function ()
       SetDocumentModified(editor:GetId(), false)
     end)
 
   editor:Connect(wxstc.wxEVT_STC_SAVEPOINTLEFT,
-    function (event)
+    function ()
       SetDocumentModified(editor:GetId(), true)
     end)
 
   editor:Connect(wxstc.wxEVT_STC_UPDATEUI,
-    function (event)
+    function ()
       updateStatusText(editor)
       updateBraceMatch(editor)
-      for e,iv in ipairs(editor.ev) do
+      for _,iv in ipairs(editor.ev) do
         local line = editor:LineFromPosition(iv[1])
         IndicateFunctions(editor,line,line+iv[2])
+        if MarkupStyle then MarkupStyle(editor,line,line+iv[2]+1) end
       end
+      if MarkupStyleRefresh then MarkupStyleRefresh(editor, editor.ev) end
       editor.ev = {}
     end)
 
+  editor:Connect(wx.wxEVT_LEFT_DOWN,
+    function (event)
+      if MarkupHotspotClick then
+        local position = editor:PositionFromPointClose(event:GetX(),event:GetY())
+        if position ~= wxstc.wxSTC_INVALID_POSITION then
+          if MarkupHotspotClick(position, editor) then return end
+        end
+      end
+      event:Skip()
+    end)
+
+  local inhandler = false
   editor:Connect(wx.wxEVT_SET_FOCUS,
     function (event)
       event:Skip()
-      if ide.in_evt_focus or exitingProgram then return end
-      ide.in_evt_focus = true
+      if inhandler or ide.exitingProgram then return end
+      inhandler = true
       isFileAlteredOnDisk(editor)
-      ide.in_evt_focus = false
+      inhandler = false
     end)
 
-  --[[
-  editor:Connect(wxstc.wxEVT_STC_POSCHANGED,
+  editor:Connect(wx.wxEVT_KEY_DOWN,
     function (event)
-      -- brace checking
-
+      local keycode = event:GetKeyCode()
+      local first, last = 0, notebook:GetPageCount()-1
+      if keycode == wx.WXK_ESCAPE and frame:IsFullScreen() then
+        ShowFullScreen(false)
+      elseif event:ControlDown() and
+        (keycode == wx.WXK_PAGEUP or keycode == wx.WXK_TAB and event:ShiftDown()) then
+        if notebook:GetSelection() == first
+        then notebook:SetSelection(last)
+        else notebook:AdvanceSelection(false) end
+      elseif event:ControlDown() and
+        (keycode == wx.WXK_PAGEDOWN or keycode == wx.WXK_TAB) then
+        if notebook:GetSelection() == last
+        then notebook:SetSelection(first)
+        else notebook:AdvanceSelection(true) end
+      else
+        if ide.osname == 'Macintosh' and event:CmdDown() then
+          return -- ignore a key press if Command key is also pressed
+        end
+        event:Skip()
+      end
     end)
-  ]]
+
   if notebook:AddPage(editor, name, true) then
     local id = editor:GetId()
     local document = {}
@@ -420,10 +544,10 @@ function GetSpec(ext,forcespec)
   -- search proper spec
   -- allow forcespec for "override"
   if ext and not spec then
-    for i,curspec in pairs(ide.specs) do
-      exts = curspec.exts
+    for _,curspec in pairs(ide.specs) do
+      local exts = curspec.exts
       if (exts) then
-        for n,curext in ipairs(exts) do
+        for _,curext in ipairs(exts) do
           if (curext == ext) then
             spec = curspec
             break
@@ -441,24 +565,17 @@ end
 function IndicateFunctions(editor, lines, linee)
   if (not (edcfg.showfncall and editor.spec and editor.spec.isfncall)) then return end
 
-  --DisplayOutput("indicate: "..tostring(lines).." "..tostring(linee).."\n")
-
   local es = editor:GetEndStyled()
   local lines = lines or 0
   local linee = linee or editor:GetLineCount()-1
 
   if (lines < 0) then return end
 
-  local isfunc = editor.spec.isfncall
-  local iscomment = editor.spec.iscomment
-  local iskeyword0 = editor.spec.iskeyword0
+  local isfncall = editor.spec.isfncall
   local isinvalid = {}
-  for i,v in pairs(iscomment) do
-    isinvalid[i] = v
-  end
-  for i,v in pairs(iskeyword0) do
-    isinvalid[i] = v
-  end
+  for i,v in pairs(editor.spec.iscomment) do isinvalid[i] = v end
+  for i,v in pairs(editor.spec.iskeyword0) do isinvalid[i] = v end
+  for i,v in pairs(editor.spec.isstring) do isinvalid[i] = v end
 
   local INDICS_MASK = wxstc.wxSTC_INDICS_MASK
   local INDIC0_MASK = wxstc.wxSTC_INDIC0_MASK
@@ -475,20 +592,13 @@ function IndicateFunctions(editor, lines, linee)
     while from do
       tx = from==1 and tx or string.sub(tx,from)
 
-      local f,t,w = isfunc(tx)
+      local f,t,w = isfncall(tx)
 
       if (f) then
         local p = ls+f+off
         local s = bit.band(editor:GetStyleAt(p),31)
-        if (not (isinvalid[s])) then
-
-          editor:StartStyling(p,INDICS_MASK)
-          editor:SetStyling(t-f,INDIC0_MASK + 1)
-        else
-          editor:StartStyling(p,INDICS_MASK)
-          editor:SetStyling(t-f,0)
-        end
-
+        editor:StartStyling(p,INDICS_MASK)
+        editor:SetStyling(#w,isinvalid[s] and 0 or (INDIC0_MASK + 1))
         off = off + t
       end
       from = t and (t+1)
@@ -500,7 +610,6 @@ end
 function SetupKeywords(editor, ext, forcespec, styles, font, fontitalic)
   local lexerstyleconvert = nil
   local spec = forcespec or GetSpec(ext)
-  --print(ext..":"..tostring(spec.apitype))
   -- found a spec setup lexers and keywords
   if spec then
     editor:SetLexer(spec.lexer or wxstc.wxSTC_LEX_NULL)
@@ -516,11 +625,11 @@ function SetupKeywords(editor, ext, forcespec, styles, font, fontitalic)
       -- Get the items in the global "wx" table for autocompletion
       if not wxkeywords then
         local keyword_table = {}
-        for index, value in pairs(wx) do
+        for index in pairs(wx) do
           table.insert(keyword_table, "wx."..index.." ")
         end
 
-        for index, value in pairs(wxstc) do
+        for index in pairs(wxstc) do
           table.insert(keyword_table, "wxstc."..index.." ")
         end
 
@@ -542,7 +651,7 @@ function SetupKeywords(editor, ext, forcespec, styles, font, fontitalic)
   end
 
   StylesApplyToEditor(styles or ide.config.styles, editor,
-    font or ide.font,fontitalic or ide.fontItalic,lexerstyleconvert)
+    font or ide.font.eNormal,fontitalic or ide.font.eItalic,lexerstyleconvert)
 end
 
 ----------------------------------------------------
@@ -570,9 +679,13 @@ funclist:Connect(wx.wxEVT_SET_FOCUS,
     local linee = editor:GetLineCount()-1
     for line=lines,linee do
       local tx = editor:GetLine(line)
-      local s,e,cap,l = editor.spec.isfndef(tx)
+      local s,_,cap,l = editor.spec.isfndef(tx)
       if (s) then
-        funclist:Append((l and "  " or "")..cap,line)
+        local ls = editor:PositionFromLine(line)
+        local style = bit.band(editor:GetStyleAt(ls+s),31)
+        if not (editor.spec.iscomment[style] or editor.spec.isstring[style]) then
+          funclist:Append((l and "  " or "")..cap,line)
+        end
       end
     end
 

src/editor/filetree.lua

@@ -23,25 +23,22 @@ ide.filetree = {
     root_id = nil,
     rootdir = "",
   },
-  imglist,
-
-  newfiledir,
 }
 local filetree = ide.filetree
-local frame = ide.frame
 
 -- generic tree
 -- ------------
 
 do
+  local getBitmap = (ide.app.createbitmap or wx.wxArtProvider.GetBitmap)
   local size = wx.wxSize(16, 16)
   filetree.imglist = wx.wxImageList(16,16)
   -- 0 = directory
-  filetree.imglist:Add(wx.wxArtProvider.GetBitmap(wx.wxART_FOLDER, wx.wxART_OTHER, size))
+  filetree.imglist:Add(getBitmap(wx.wxART_FOLDER, wx.wxART_OTHER, size))
   -- 1 = file known spec
-  filetree.imglist:Add(wx.wxArtProvider.GetBitmap(wx.wxART_HELP_PAGE, wx.wxART_OTHER, size))
+  filetree.imglist:Add(getBitmap(wx.wxART_HELP_PAGE, wx.wxART_OTHER, size))
   -- 2 = file rest
-  filetree.imglist:Add(wx.wxArtProvider.GetBitmap(wx.wxART_NORMAL_FILE, wx.wxART_OTHER, size))
+  filetree.imglist:Add(getBitmap(wx.wxART_NORMAL_FILE, wx.wxART_OTHER, size))
 end
 
 local function treeAddDir(tree,parent_id,rootdir)
@@ -50,15 +47,14 @@ local function treeAddDir(tree,parent_id,rootdir)
   while true do
     if not item:IsOk() then break end
     items[tree:GetItemText(item) .. tree:GetItemImage(item)] = item
-    item, cookie = tree:GetNextChild(item, cookie)
+    item, cookie = tree:GetNextChild(parent_id, cookie)
   end
 
   local curr
-  local search = rootdir..string_Pathsep.."*.*"
-  local dirs = FileSysGet(search,wx.wxDIR)
+  local search = rootdir..string_Pathsep.."*"
 
   -- append directories
-  for i,dir in ipairs(dirs) do
+  for _,dir in ipairs(FileSysGet(search,wx.wxDIR)) do
     local name = dir:match("%"..string_Pathsep.."("..stringset_File.."+)$")
     local icon = 0
     local item = items[name .. icon]
@@ -82,10 +78,9 @@ local function treeAddDir(tree,parent_id,rootdir)
   end
 
   -- then append files
-  local files = FileSysGet(search,wx.wxFILE)
-  for i,file in ipairs(files) do
+  for _,file in ipairs(FileSysGet(search,wx.wxFILE)) do
     local name = file:match("%"..string_Pathsep.."("..stringset_File.."+)$")
-    local known = GetSpec(GetFileExt(fname))
+    local known = GetSpec(GetFileExt(name))
     local icon = known and 1 or 2
     local item = items[name .. icon]
     if item then -- existing item
@@ -126,15 +121,17 @@ local function treeGetItemFullName(tree,treedata,item_id)
     cur = tree:GetItemText(item_id)
     if cur and string.len(cur) > 0 then str = cur..string_Pathsep..str end
   end
-  return ((not filetree.showroot) and filetree.projdata.rootdir or "")..str
+  -- as root may already include path separate, normalize the path
+  local fullPath = wx.wxFileName(
+    filetree.showroot and str or filetree.projdata.rootdir .. str)
+  fullPath:Normalize()
+  return fullPath:GetFullPath()
 end
 
 local function treeSetRoot(tree,treedata,rootdir)
   tree:DeleteAllItems()
-
   if (not wx.wxDirExists(rootdir)) then
     treedata.root_id = nil
-    tree:AddRoot("Invalid")
     return
   end
 
@@ -160,10 +157,7 @@ local function treeSetConnectorsAndIcons(tree,treedata)
       return true
     end )
   tree:Connect( wx.wxEVT_COMMAND_TREE_ITEM_COLLAPSED,
-    function( event )
-      -- don't need to do anything here
-      return true
-    end )
+    function() return true end )
   tree:Connect( wx.wxEVT_COMMAND_TREE_ITEM_ACTIVATED,
     function( event )
       local item_id = event:GetItem()
@@ -179,27 +173,11 @@ local function treeSetConnectorsAndIcons(tree,treedata)
       else -- open file
         if wx.wxFileName(name):FileExists() then
           LoadFile(name,nil,true)
-          FileTreeMarkSelected(name)
         else -- stale filetree information; rescan
           treeAddDir(tree,tree:GetItemParent(item_id),name)
         end 
       end
     end )
-  tree:Connect( wx.wxEVT_COMMAND_TREE_SEL_CHANGED,
-    function( event )
-      local item_id = event:GetItem()
-
-      -- set "newfile-path"
-      local isfile = tree:GetItemImage(item_id) ~= 0
-      filetree.newfiledir = treeGetItemFullName(tree,treedata,item_id)
-
-      if (isfile) then
-        -- remove file
-        filetree.newfiledir = wx.wxFileName(filetree.newfiledir):GetPath(wx.wxPATH_GET_VOLUME)
-      end
-
-      filetree.newfiledir = filetree.newfiledir..string_Pathsep
-    end )
 end
 
 -- project
@@ -221,7 +199,8 @@ local projtree = wx.wxTreeCtrl(projpanel, ID "filetree.projtree",
   or (wx.wxTR_HAS_BUTTONS + wx.wxTR_SINGLE + wx.wxTR_HIDE_ROOT))
 
 -- use the same font in the combobox as is used in the filetree
-projcombobox:SetFont(projtree:GetFont())
+projtree:SetFont(ide.font.fNormal)
+projcombobox:SetFont(ide.font.fNormal)
 
 local projTopSizer = wx.wxBoxSizer( wx.wxHORIZONTAL );
 projTopSizer:Add(projcombobox, 1, wx.wxALL + wx.wxALIGN_LEFT + wx.wxGROW, 0)
@@ -287,8 +266,8 @@ projpanel.projcombobox = projcombobox
 projpanel.projtree = projtree
 
 function FileTreeGetDir()
-  -- atm only projtree
-  return projpanel:IsShown() and (filetree.newfiledir .. string_Pathsep)
+  return projpanel:IsShown() and filetree.newfiledir
+    and wx.wxFileName.DirName(filetree.newfiledir):GetFullPath()
 end
 
 function FileTreeSetProjects(tab)
@@ -306,7 +285,7 @@ local function findItem(tree, match)
   local node = projtree:GetRootItem()
   local label = tree:GetItemText(node)
 
-  local s, e = string.find(match, label .. string_Pathsep)
+  local s, e = string.find(match, label, 1, true)
   if not s or s ~= 1 then return end
 
   for token in string.gmatch(string.sub(match,e+1), "[^%"..string_Pathsep.."]+") do
@@ -320,7 +299,7 @@ local function findItem(tree, match)
         node = item
         break
       end
-      item, cookie = tree:GetNextChild(item, cookie)
+      item, cookie = tree:GetNextChild(node, cookie)
     end
   end
 
@@ -328,18 +307,32 @@ local function findItem(tree, match)
   return node
 end
 
-local curr_id
+local curr_file
 function FileTreeMarkSelected(file)
-  if not file then return end
+  if not file or not filetree.projdirText or #filetree.projdirText == 0 then return end
+
   local item_id = findItem(projtree, file)
-  if curr_id ~= item_id then
-    if curr_id and projtree:IsBold(curr_id) then
-      projtree:SetItemBold(curr_id, false)
+
+  -- if the select item is different from the current one
+  -- or the current one is the same, but not bold (which may happen when
+  -- the project is changed to one that includes the current item)
+  if curr_file ~= file
+  or item_id and not projtree:IsBold(item_id) then
+    if curr_file then
+      local curr_id = findItem(projtree, curr_file)
+      if curr_id and projtree:IsBold(curr_id) then
+        projtree:SetItemBold(curr_id, false)
+      end
     end
     if item_id then
       projtree:EnsureVisible(item_id)
       projtree:SetItemBold(item_id, true)
-      curr_id = item_id
     end
+    curr_file = file
+    projtree:Refresh() -- to force refresh on Mac (ide.osname == 'Macintosh')
   end
 end
+
+function FileTreeRefresh()
+  treeSetRoot(projtree,filetree.projdata,filetree.projdirText)
+end

src/editor/findreplace.lua

@@ -112,7 +112,7 @@ function findReplace:FindString(reverse)
     end
     if posFind == -1 then
       findReplace.foundString = false
-      ide.frame:SetStatusText("Find text not found.")
+      ide.frame:SetStatusText("Text not found.")
     else
       findReplace.foundString = true
       local start = editor:GetTargetStart()
@@ -209,48 +209,36 @@ local function onFileRegister(pos)
   findReplace.occurrences = findReplace.occurrences + 1
 end
 
-local function storefile(file,content)
-  local handle = io.open(file, "wb")
-  if (handle) then
-    handle:write(content)
-    handle:close()
-  end
-end
-
 local function ProcInFiles(startdir,mask,subdirs,replace)
   if (subdirs) then
-    local dirs = FileSysGet(startdir..string_Pathsep.."*.*",wx.wxDIR)
-    for i,dir in ipairs(dirs) do
-      --DisplayOutput(dir.."\n")
+    local dirs = FileSysGet(startdir..string_Pathsep.."*",wx.wxDIR)
+    for _,dir in ipairs(dirs) do
       ProcInFiles(dir,mask,true,replace)
     end
   end
 
   local files = FileSysGet(startdir..string_Pathsep..mask,wx.wxFILE)
-  for i,file in ipairs(files) do
-    findReplace.curfilename = file
+  for _,file in ipairs(files) do
+    -- ignore .bak files when replacing and asked to store .bak files
+    if not (replace and findReplace.fMakeBak and file:find('.bak$')) then
+      findReplace.curfilename = file
 
-    --DisplayOutput(file.."\n")
+      local filetext = FileRead(file)
+      if filetext then
+        findReplace.oveditor:SetText(filetext)
 
-    -- load file
-    local handle = io.open(file, "rb")
-    if handle then
-      local filetext = handle:read("*a")
-      handle:close()
-      findReplace.oveditor:SetText(filetext)
-
-      if (replace and findReplace:ReplaceString(true,onFileRegister)) then
-        -- store changed content, make .bak
-        if (findReplace.fMakeBak) then
-          storefile(file..".bak",filetext)
+        if replace then
+          -- check if anything replaced, store changed content, make .bak
+          if findReplace:ReplaceString(true,onFileRegister)
+          and findReplace.fMakeBak and FileWrite(file..".bak",filetext) then
+            FileWrite(file,findReplace.oveditor:GetText())
+          end
+        else
+          findReplace:FindStringAll(onFileRegister)
         end
-        storefile(file,findReplace.oveditor:GetText())
-      else
-        findReplace:FindStringAll(onFileRegister)
       end
     end
   end
-
 end
 
 function findReplace:RunInFiles(replace)
@@ -265,11 +253,12 @@ function findReplace:RunInFiles(replace)
 
   local fname = wx.wxFileName(findReplace.filedirText)
   local startdir = findReplace.filedirText
-  DisplayOutput("> FindInFiles "..(replace and "Replacing" or "Searching")..": '"..findReplace.findText.."'\n")
+  DisplayOutput("FindInFiles: "..(replace and "Replacing" or "Searching for").." '"..findReplace.findText.."'.\n")
 
-  ProcInFiles(startdir, findReplace.filemaskText,findReplace.fSubDirs, replace)
+  ProcInFiles(startdir, findReplace.filemaskText, findReplace.fSubDirs, replace)
 
-  DisplayOutput("> FindInFiles: "..findReplace.occurrences.." occurrence(s) have been found\n\n")
+  DisplayOutput("FindInFiles: "..findReplace.occurrences.." instance(s) have been "..
+    (replace and "replaced" or "found")..".\n")
   findReplace.oveditor = nil
 end
 
@@ -440,7 +429,7 @@ local function createFindReplaceDialog(replace,infiles)
   end
 
   findDialog:Connect(ID_FIND_NEXT, wx.wxEVT_COMMAND_BUTTON_CLICKED,
-    function(event)
+    function()
       TransferDataFromWindow()
       if (findReplace.infiles) then
         findReplace:RunInFiles()
@@ -479,8 +468,8 @@ local function createFindReplaceDialog(replace,infiles)
 
   if infilesDirButton then
     findDialog:Connect(ID_SETDIR, wx.wxEVT_COMMAND_BUTTON_CLICKED,
-      function(event)
-        local filePicker = wx.wxDirDialog(findDialog, "Chose a project directory",
+      function()
+        local filePicker = wx.wxDirDialog(findDialog, "Choose a project directory",
           findReplace.filedirText~="" and findReplace.filedirText or wx.wxGetCwd(),wx.wxFLP_USE_TEXTCTRL)
 
         local res = filePicker:ShowModal(true)

src/editor/gui.lua

@@ -10,33 +10,23 @@ BREAKPOINT_MARKER_VALUE = 2 -- = 2^BREAKPOINT_MARKER
 CURRENT_LINE_MARKER = 2
 CURRENT_LINE_MARKER_VALUE = 4 -- = 2^CURRENT_LINE_MARKER
 
--- Globals
-local font = nil -- fonts to use for the editor
-local fontItalic = nil
-local ofont = nil -- fonts to use for the outputshell
-local ofontItalic = nil
--- ----------------------------------------------------------------------------
 -- Pick some reasonable fixed width fonts to use for the editor
-if wx.__WXMSW__ then
-  font = wx.wxFont(ide.config.editor.fontsize or 10, wx.wxFONTFAMILY_MODERN, wx.wxFONTSTYLE_NORMAL, wx.wxFONTWEIGHT_NORMAL, false, ide.config.editor.fontname or "Courier New")
-  fontItalic = wx.wxFont(ide.config.editor.fontsize or 10, wx.wxFONTFAMILY_MODERN, wx.wxFONTSTYLE_ITALIC, wx.wxFONTWEIGHT_NORMAL, false, ide.config.editor.fontname or "Courier New")
-else
-  font = wx.wxFont(ide.config.editor.fontsize or 10, wx.wxFONTFAMILY_MODERN, wx.wxFONTSTYLE_NORMAL, wx.wxFONTWEIGHT_NORMAL, false, ide.config.editor.fontname or "")
-  fontItalic = wx.wxFont(ide.config.editor.fontsize or 10, wx.wxFONTFAMILY_MODERN, wx.wxFONTSTYLE_ITALIC, wx.wxFONTWEIGHT_NORMAL, false, ide.config.editor.fontname or "")
+local function setFont(style, config, name, size)
+  return wx.wxFont(config.fontsize or size or 10, wx.wxFONTFAMILY_MODERN, style,
+    wx.wxFONTWEIGHT_NORMAL, false, config.fontname or name,
+    config.fontencoding or wx.wxFONTENCODING_DEFAULT)
 end
+ide.font.eNormal = setFont(wx.wxFONTSTYLE_NORMAL, ide.config.editor, wx.__WXMSW__ and "Courier New" or "")
+ide.font.eItalic = setFont(wx.wxFONTSTYLE_ITALIC, ide.config.editor, wx.__WXMSW__ and "Courier New" or "")
 
-if wx.__WXMSW__ then
-  ofont = wx.wxFont(ide.config.outputshell.fontsize or 10, wx.wxFONTFAMILY_MODERN, wx.wxFONTSTYLE_NORMAL, wx.wxFONTWEIGHT_NORMAL, false, ide.config.outputshell.fontname or "Courier New")
-  ofontItalic = wx.wxFont(ide.config.outputshell.fontsize or 10, wx.wxFONTFAMILY_MODERN, wx.wxFONTSTYLE_ITALIC, wx.wxFONTWEIGHT_NORMAL, false, ide.config.outputshell.fontname or "Courier New")
-else
-  ofont = wx.wxFont(ide.config.outputshell.fontsize or 10, wx.wxFONTFAMILY_MODERN, wx.wxFONTSTYLE_NORMAL, wx.wxFONTWEIGHT_NORMAL, false, ide.config.outputshell.fontname or "")
-  ofontItalic = wx.wxFont(ide.config.outputshell.fontsize or 10, wx.wxFONTFAMILY_MODERN, wx.wxFONTSTYLE_ITALIC, wx.wxFONTWEIGHT_NORMAL, false, ide.config.outputshell.fontname or "")
-end
+ide.font.oNormal = setFont(wx.wxFONTSTYLE_NORMAL, ide.config.outputshell, wx.__WXMSW__ and "Courier New" or "")
+ide.font.oItalic = setFont(wx.wxFONTSTYLE_ITALIC, ide.config.outputshell, wx.__WXMSW__ and "Courier New" or "")
 
-ide.font = font
-ide.fontItalic = fontItalic
-ide.ofont = ofont
-ide.ofontItalic = ofontItalic
+-- treeCtrl font requires slightly different handling
+local gui, config = wx.wxTreeCtrl():GetFont(), ide.config.filetree
+if config.fontsize then gui:SetPointSize(config.fontsize) end
+if config.fontname then gui:SetFaceName(config.fontname) end
+ide.font.fNormal = gui
 
 -- ----------------------------------------------------------------------------
 -- Create the wxFrame
@@ -44,7 +34,10 @@ ide.ofontItalic = ofontItalic
 local function createFrame()
   frame = wx.wxFrame(wx.NULL, wx.wxID_ANY, GetIDEString("editor"),
     wx.wxDefaultPosition, wx.wxSize(1000, 700))
-  frame:DragAcceptFiles(true)
+  -- wrap into protected call as DragAcceptFiles fails on MacOS with
+  -- wxwidgets 2.8.12 even though it should work according to change notes
+  -- for 2.8.10: "Implemented wxWindow::DragAcceptFiles() on all platforms."
+  pcall(function() frame:DragAcceptFiles(true) end)
   frame:Connect(wx.wxEVT_DROP_FILES,function(evt)
       local files = evt:GetFiles()
       if not files or #files == 0 then return end
@@ -54,19 +47,21 @@ local function createFrame()
     end)
 
   local menuBar = wx.wxMenuBar()
-  frame:SetMenuBar(menuBar)
-  frame.menuBar = menuBar
-  
-  local statusBar = frame:CreateStatusBar( 5 )
-  frame.statusBar = statusBar
-  local status_txt_width = statusBar:GetTextExtent("OVRW")
-  statusBar:SetStatusWidths({-1, status_txt_width*6, status_txt_width, status_txt_width, status_txt_width*5})
+  local statusBar = frame:CreateStatusBar(6)
+  local section_width = statusBar:GetTextExtent("OVRW")
+  statusBar:SetStatusStyles({wx.wxSB_RAISED, wx.wxSB_RAISED, wx.wxSB_RAISED,
+    wx.wxSB_RAISED, wx.wxSB_RAISED, wx.wxSB_RAISED})
+  statusBar:SetStatusWidths(
+    {-1, section_width*6, section_width, section_width, section_width*4, section_width*4})
   statusBar:SetStatusText(GetIDEString("statuswelcome"))
   
   local mgr = wxaui.wxAuiManager()
-  frame.uimgr = mgr
   mgr:SetManagedWindow(frame)
 
+  frame.menuBar = menuBar
+  frame.statusBar = statusBar
+  frame.uimgr = mgr
+
   return frame
 end
 
@@ -76,23 +71,24 @@ local function createToolBar(frame)
   local funclist = wx.wxChoice.new(toolBar,ID "toolBar.funclist",wx.wxDefaultPosition, wx.wxSize.new(240,16))
   
   -- note: Ususally the bmp size isn't necessary, but the HELP icon is not the right size in MSW
+  local getBitmap = (ide.app.createbitmap or wx.wxArtProvider.GetBitmap)
   local toolBmpSize = toolBar:GetToolBitmapSize()
-  toolBar:AddTool(ID_NEW, "New", wx.wxArtProvider.GetBitmap(wx.wxART_NORMAL_FILE, wx.wxART_TOOLBAR, toolBmpSize), "Create an empty document")
-  toolBar:AddTool(ID_OPEN, "Open", wx.wxArtProvider.GetBitmap(wx.wxART_FILE_OPEN, wx.wxART_TOOLBAR, toolBmpSize), "Open an existing document")
-  toolBar:AddTool(ID_SAVE, "Save", wx.wxArtProvider.GetBitmap(wx.wxART_FILE_SAVE, wx.wxART_TOOLBAR, toolBmpSize), "Save the current document")
-  toolBar:AddTool(ID_SAVEALL, "Save All", wx.wxArtProvider.GetBitmap(wx.wxART_NEW_DIR, wx.wxART_TOOLBAR, toolBmpSize), "Save all documents")
+  toolBar:AddTool(ID_NEW, "New", getBitmap(wx.wxART_NORMAL_FILE, wx.wxART_TOOLBAR, toolBmpSize), "Create an empty document")
+  toolBar:AddTool(ID_OPEN, "Open", getBitmap(wx.wxART_FILE_OPEN, wx.wxART_TOOLBAR, toolBmpSize), "Open an existing document")
+  toolBar:AddTool(ID_SAVE, "Save", getBitmap(wx.wxART_FILE_SAVE, wx.wxART_TOOLBAR, toolBmpSize), "Save the current document")
+  toolBar:AddTool(ID_SAVEALL, "Save All", getBitmap(wx.wxART_NEW_DIR, wx.wxART_TOOLBAR, toolBmpSize), "Save all documents")
   toolBar:AddSeparator()
-  toolBar:AddTool(ID_CUT, "Cut", wx.wxArtProvider.GetBitmap(wx.wxART_CUT, wx.wxART_TOOLBAR, toolBmpSize), "Cut the selection")
-  toolBar:AddTool(ID_COPY, "Copy", wx.wxArtProvider.GetBitmap(wx.wxART_COPY, wx.wxART_TOOLBAR, toolBmpSize), "Copy the selection")
-  toolBar:AddTool(ID_PASTE, "Paste", wx.wxArtProvider.GetBitmap(wx.wxART_PASTE, wx.wxART_TOOLBAR, toolBmpSize), "Paste text from the clipboard")
+  toolBar:AddTool(ID_CUT, "Cut", getBitmap(wx.wxART_CUT, wx.wxART_TOOLBAR, toolBmpSize), "Cut the selection")
+  toolBar:AddTool(ID_COPY, "Copy", getBitmap(wx.wxART_COPY, wx.wxART_TOOLBAR, toolBmpSize), "Copy the selection")
+  toolBar:AddTool(ID_PASTE, "Paste", getBitmap(wx.wxART_PASTE, wx.wxART_TOOLBAR, toolBmpSize), "Paste text from the clipboard")
   toolBar:AddSeparator()
-  toolBar:AddTool(ID_UNDO, "Undo", wx.wxArtProvider.GetBitmap(wx.wxART_UNDO, wx.wxART_TOOLBAR, toolBmpSize), "Undo last edit")
-  toolBar:AddTool(ID_REDO, "Redo", wx.wxArtProvider.GetBitmap(wx.wxART_REDO, wx.wxART_TOOLBAR, toolBmpSize), "Redo last undo")
+  toolBar:AddTool(ID_UNDO, "Undo", getBitmap(wx.wxART_UNDO, wx.wxART_TOOLBAR, toolBmpSize), "Undo last edit")
+  toolBar:AddTool(ID_REDO, "Redo", getBitmap(wx.wxART_REDO, wx.wxART_TOOLBAR, toolBmpSize), "Redo last undo")
   toolBar:AddSeparator()
-  toolBar:AddTool(ID_FIND, "Find", wx.wxArtProvider.GetBitmap(wx.wxART_FIND, wx.wxART_TOOLBAR, toolBmpSize), "Find text")
-  toolBar:AddTool(ID_REPLACE, "Replace", wx.wxArtProvider.GetBitmap(wx.wxART_FIND_AND_REPLACE, wx.wxART_TOOLBAR, toolBmpSize), "Find and replace text")
+  toolBar:AddTool(ID_FIND, "Find", getBitmap(wx.wxART_FIND, wx.wxART_TOOLBAR, toolBmpSize), "Find text")
+  toolBar:AddTool(ID_REPLACE, "Replace", getBitmap(wx.wxART_FIND_AND_REPLACE, wx.wxART_TOOLBAR, toolBmpSize), "Find and replace text")
   toolBar:AddSeparator()
-  toolBar:AddTool(ID "debug.projectdir.fromfile", "Update", wx.wxArtProvider.GetBitmap(wx.wxART_GO_DIR_UP , wx.wxART_TOOLBAR, toolBmpSize), "Sets projectdir from file")
+  toolBar:AddTool(ID "debug.projectdir.fromfile", "Update", getBitmap(wx.wxART_GO_DIR_UP , wx.wxART_TOOLBAR, toolBmpSize), "Sets projectdir from file")
   toolBar:AddSeparator()
   toolBar:AddControl(funclist)
   toolBar:Realize()
@@ -110,26 +106,27 @@ local function createNotebook(frame)
   wxaui.wxAUI_NB_DEFAULT_STYLE + wxaui.wxAUI_NB_TAB_EXTERNAL_MOVE
   + wx.wxNO_BORDER)
 
+  -- the following group of event handlers allows the active editor
+  -- to get/keep focus after execution of Run and other commands
   local current -- the currently active editor, needed by the focus selection
   local function onPageChange(event)
     current = event:GetSelection() -- update the active editor reference
     SetEditorSelection(current)
     event:Skip() -- skip to let page change
   end
-
+  notebook:Connect(wx.wxEVT_SET_FOCUS, -- Notepad tabs shouldn't be selectable,
+    function (event)
+      SetEditorSelection(current) -- select the currently active editor
+    end)
   notebook:Connect(wx.wxEVT_COMMAND_NOTEBOOK_PAGE_CHANGED, onPageChange)
   notebook:Connect(wxaui.wxEVT_COMMAND_AUINOTEBOOK_PAGE_CHANGED, onPageChange)
+
   notebook:Connect(wxaui.wxEVT_COMMAND_AUINOTEBOOK_PAGE_CLOSE,
     function (event)
       ClosePage(event:GetSelection())
       event:Veto() -- don't propagate the event as the page is already closed
     end)
 
-  notebook:Connect(wx.wxEVT_SET_FOCUS, -- Notepad tabs shouldn't be selectable,
-    function (event)
-      SetEditorSelection(current) -- select the currently active editor
-    end)
-  
   frame.notebook = notebook
   return notebook
 end

src/editor/ids.lua

@@ -37,14 +37,13 @@ ID_FIND_IN_FILES    = NewID()
 ID_REPLACE_IN_FILES = NewID()
 ID_GOTOLINE         = NewID()
 ID_SORT             = NewID()
--- Debug menu
+-- Project/Debug menu
 ID_TOGGLEBREAKPOINT = NewID()
 ID_COMPILE          = NewID()
 ID_RUN              = NewID()
+ID_RUNNOW           = NewID()
 ID_ATTACH_DEBUG     = NewID()
 ID_START_DEBUG      = NewID()
---ID_USECONSOLE       = NewID()
-
 ID_STOP_DEBUG       = NewID()
 ID_STEP             = NewID()
 ID_STEP_OVER        = NewID()
@@ -52,10 +51,12 @@ ID_STEP_OUT         = NewID()
 ID_CONTINUE         = NewID()
 ID_BREAK            = NewID()
 ID_TRACE            = NewID()
---ID_VIEWCALLSTACK    = NewID()
---ID_VIEWWATCHWINDOW  = NewID()
+ID_VIEWCALLSTACK    = NewID()
+ID_VIEWWATCHWINDOW  = NewID()
+ID_FULLSCREEN       = NewID()
 ID_CLEAROUTPUT      = NewID()
-ID_DEBUGGER_PORT    = NewID()
+ID_PROJECTDIR       = NewID()
+ID_INTERPRETER      = NewID()
 -- Help menu
 ID_ABOUT            = wx.wxID_ABOUT
 -- Watch window menu items
@@ -65,8 +66,6 @@ ID_EDITWATCH        = NewID()
 ID_REMOVEWATCH      = NewID()
 ID_EVALUATEWATCH    = NewID()
 
-ID_WORKDIR_CHOSE = NewID()
-
 local ids = {}
 function ID (name)
 	ids[name] = ids[name] or NewID()

src/editor/inspect.lua

@@ -0,0 +1,185 @@
+-- Integration with LuaInspect
+-- (C) 2012 Paul Kulchenko
+
+local M, LA, LI, T = {}
+local FAST = true
+
+local function init()
+  if LA then return end
+
+  require "metalua"
+  LA = require "luainspect.ast"
+  LI = require "luainspect.init"
+  T = require "luainspect.types"
+
+  if FAST then
+    LI.eval_comments = function () end
+    LI.infer_values = function () end
+  end
+end
+
+function M.warnings_from_string(src, file)
+  init()
+
+  local ast, err, linenum, colnum = LA.ast_from_string(src, file)
+  if err then return nil, err, linenum, colnum end
+
+  if FAST then
+    LI.inspect(ast, nil, src)
+    LA.ensure_parents_marked(ast)
+  else
+    local tokenlist = LA.ast_to_tokenlist(ast, src)
+    LI.inspect(ast, tokenlist, src)
+    LI.mark_related_keywords(ast, tokenlist, src)
+  end
+
+  return M.show_warnings(ast)
+end
+
+function M.show_warnings(top_ast)
+  local warnings = {}
+  local function warn(msg, linenum, path)
+    warnings[#warnings+1] = (path or "?") .. "(" .. (linenum or 0) .. "): " .. msg
+  end
+  local function known(o) return not T.istype[o] end
+  local isseen, globseen = {}, {}
+  LA.walk(top_ast, function(ast)
+    local line = ast.lineinfo and ast.lineinfo.first[1] or 0
+    local path = ast.lineinfo and ast.lineinfo.first[4] or '?'
+    local name = ast[1]
+    -- check if we're masking a variable in the same scope
+    if ast.localmasking and name ~= '_' and
+       ast.level == ast.localmasking.level then
+      local linenum = ast.localmasking.lineinfo.first[1]
+      local parent = ast.parent and ast.parent.parent
+      local func = parent and parent.tag == 'Localrec'
+      warn("local " .. (func and 'function' or 'variable') .. " '" ..
+        name .. "' masks earlier declaration " ..
+        (linenum and "on line " .. linenum or "in the same scope"),
+        line, path)
+    end
+    if ast.localdefinition == ast and not ast.isused and
+       not ast.isignore then
+      local parent = ast.parent and ast.parent.parent
+      local isparam = parent and parent.tag == 'Function'
+      if isparam then
+        if name ~= 'self' then
+          local func = parent.parent and parent.parent.parent
+          local assignment = not func.tag or func.tag == 'Set' or func.tag == 'Localrec'
+          local func1 = func[1][1]
+          local fname = assignment and func1 and type(func1[1]) == 'string' and func1[1]
+            or (func1.tag == 'Index' and func1[1][1] .. '.' .. func1[2][1])
+          -- "function foo(bar)" => func.tag == 'Set'
+          --   `Set{{`Id{"foo"}},{`Function{{`Id{"bar"}},{}}}}
+          -- "local function foo(bar)" => func.tag == 'Localrec'
+          -- "local _, foo = 1, function(bar)" => func.tag == 'Local'
+          -- "print(function(bar) end)" => func.tag == nil
+          -- "function tbl:foo(bar)" => func.tag == 'Set'
+          --   `Set{{`Index{`Id{"tbl"},`String{"foo"}}},{`Function{{`Id{"self"},`Id{"bar"}},{}}}}
+          warn("unused parameter '" .. name .. "'" ..
+               (func and assignment
+                     and (fname and func.tag
+                               and (" in function '" .. fname .. "'")
+                               or " in anonymous function")
+                     or ""),
+               line, path)
+        end
+      else
+        if parent.tag == 'Localrec' then -- local function foo...
+          warn("unused local function '" .. name .. "'", line, path)
+        else
+          warn("unused local variable '" .. name .. "'; "..
+               "consider removing or replacing with '_'", line, path)
+        end
+      end
+    end
+    -- added check for FAST as ast.seevalue relies on value evaluation,
+    -- which is very slow even on simple and short scripts
+    if not FAST and ast.isfield and not(known(ast.seevalue.value) and ast.seevalue.value ~= nil) then
+      warn("unknown field " .. name, ast.lineinfo.first[1], path)
+    elseif ast.tag == 'Id' and not ast.localdefinition and not ast.definedglobal then
+      if not globseen[name] then
+        globseen[name] = true
+        local parent = ast.parent
+        -- if being called and not one of the parameters
+        if parent and parent.tag == 'Call' and parent[1] == ast then
+          warn("first use of unknown global function '" .. name .. "'", line, path)
+        else
+          warn("first use of unknown global variable '" .. name .. "'", line, path)
+        end
+      end
+    elseif ast.tag == 'Id' and not ast.localdefinition and ast.definedglobal then
+      local parent = ast.parent and ast.parent.parent
+      if parent and parent.tag == 'Set' and not globseen[name] -- report assignments to global
+        -- only report if it is on the left side of the assignment
+        -- this is a bit tricky as it can be assigned as part of a, b = c, d
+        -- `Set{ {lhs+} {expr+} } -- lhs1, lhs2... = e1, e2...
+        and parent[1] == ast.parent
+        and parent[2][1].tag ~= "Function" then -- but ignore global functions
+        warn("first assignment to global variable '" .. name .. "'", line, path)
+        globseen[name] = true
+      end
+    elseif (ast.tag == 'Set' or ast.tag == 'Local') and #(ast[2]) > #(ast[1]) then
+      warn(("value discarded in multiple assignment: %d values assigned to %d variable%s")
+        :format(#(ast[2]), #(ast[1]), #(ast[1]) > 1 and 's' or ''), line, path)
+    end
+    local vast = ast.seevalue or ast
+    local note = vast.parent
+             and (vast.parent.tag == 'Call' or vast.parent.tag == 'Invoke')
+             and vast.parent.note
+    if note and not isseen[vast.parent] then
+      isseen[vast.parent] = true
+      warn("function '" .. name .. "': " .. note, line, path)
+    end
+  end)
+  return warnings
+end
+
+local frame = ide.frame
+local menu = frame.menuBar:GetMenu(frame.menuBar:FindMenu("&Project"))
+local ID_ANALYZE = ID "debug.analyze"
+
+-- insert after "Compile" item
+for item = 0, menu:GetMenuItemCount()-1 do
+   if menu:FindItemByPosition(item):GetId() == ID_COMPILE then
+     menu:Insert(item+1, ID_ANALYZE, "Analyze\tShift-F7", "Analyze the source code")
+     break
+   end
+end
+
+local debugger = ide.debugger
+local openDocuments = ide.openDocuments
+
+local function analyzeProgram(editor)
+  local editorText = editor:GetText()
+  local id = editor:GetId()
+  local filePath = DebuggerMakeFileName(editor, openDocuments[id].filePath)
+
+  if frame.menuBar:IsChecked(ID_CLEAROUTPUT) then ClearOutput() end
+  DisplayOutput("Analizing the source code")
+  frame:Update()
+
+  local warn, err = M.warnings_from_string(editorText, filePath)
+  if err then -- report compilation error
+    DisplayOutput(": not completed\n")
+    return false
+  end
+
+  DisplayOutput((": %s warning%s.\n")
+    :format(#warn > 0 and #warn or 'no', #warn == 1 and '' or 's'))
+  DisplayOutputNoMarker(table.concat(warn, "\n") .. "\n")
+
+  return true -- analyzed ok
+end
+
+frame:Connect(ID_ANALYZE, wx.wxEVT_COMMAND_MENU_SELECTED,
+  function ()
+    ActivateOutput()
+    local editor = GetEditor()
+    if not analyzeProgram(editor) then CompileProgram(editor) end
+  end)
+frame:Connect(ID_ANALYZE, wx.wxEVT_UPDATE_UI,
+  function (event)
+    local editor = GetEditor()
+    event:Enable((debugger.server == nil and debugger.pid == nil) and (editor ~= nil))
+  end)