Updated CHANGELOG for 0.40.

LuaDist relies on using liblua.dylib for all its modules with the same
name for both Lua 5.1 and Lua 5.2 interpreters. This presents an issue for
ZBS as it needs not only to refer to a proper liblua.dylib, but also to
avoid loading two interpreters as its own interperter loads its own
library.

To resolve this, liblua.dylib doesn't export any symbols (it's a dummy
library), but it needs to be present to satisfy dyld dependency (as some
of the libraries and LuaDist build process are outside of our control).

All the other Lua symbols are already available in memory, but to make
LuaDist libraries to use them, we enforce flat namespace.

CHANGELOG.md

@@ -1,16 +1,164 @@
 # ZeroBrane Studio Changelog
 
-## Current master (Aug 29 2013)
+## v0.40 (Dec 14 2013)
+
+### Highlights
+  - Added LuaDist integration.
+  - Added live coding support for GSL-shell.
+  - Added support for project settings.
+  - Added filetree operations.
+  - Added Busted interpreter.
+
+### Special thanks
+  - To Jayanth Acharya for SciTeLuaIDE color scheme.
+  - To Mike Richards for adding edge handling and styles.
+  - To [adamdmoss](https://github.com/adamdmoss) for Mobdebug API update.
+
+### Improvements
+  - Added live coding support for GSL-shell (closes #239).
+  - Added support for product-specific links in the Help menu.
+  - Added 'edge' style to the color schemes (ref #237).
+  - Added ability to set 'edge' style properties individually (ref #237).
+  - Add edge to styles
+  - Add fold margin checker color to styles
+  - Add edge line and fold margin checker color
+  - Added changing directory when launching on Linux (closes #157).
+  - Added setting PATH for LfW to allow loading of DLL dependencies.
+  - Added logic to set architecture dynamically for luadist (ref #225).
+  - Added luadist bootstrap dependencies for Linux (ref #225).
+  - Added option to load luadist as a module (ref #225).
+  - Added luadist bootstrap dependencies for OSX (ref #225).
+  - Added proxy dll for Lua 5.2 (ref #225).
+  - Added luadist bootstrap dependencies for Windows (ref #225).
+  - Added package GetInterpreters method (ref #166, #225).
+  - Added package AddConsoleAlias/RemoveConsoleAlias methods (ref #166, #225).
+  - Added version property to the Lua interpreters.
+  - Added new location for Marmalade Quick v7+ and s3e path logic (fixes #226).
+  - Added directory creation (if needed) to file saving.
+  - Added support for symlinks in the filetree (with recursion protection).
+  - Added package AddConfig/RemoveConfig methods (ref #166).
+  - Added package GetProject method (ref #166).
+  - Added package onProjectPreLoad method (ref #166).
+  - Added workaround for conflict with Scintilla shortcuts on Linux.
+  - Added 'Open with Default Program' to file tree operations (ref #123).
+  - Added toggling directory on Enter (ref #123).
+  - Added 'Copy Full Path' to file tree operations (ref #123).
+  - Added deleting file/directory to file tree operations (ref #123).
+  - Added processing of packages from `$HOME/.zbstudio/packages` folder (#166).
+  - Added 'New File' and 'New Directory' to file operations (ref #123).
+  - Added error reporting for failed rename operations (ref #123).
+  - Added re-opening editor tabs affected by directory move/rename (ref #123).
+  - Added package FindDocumentsByPartialPath method (ref #166).
+  - Added existing file overwrite confirmation when renaming (ref #123).
+  - Added existing file overwrite confirmation when saving.
+  - Added creating intermediate directories during file rename (ref #123).
+  - Added in-place editing of file and folder names (ref #123).
+  - Added refreshing editor tab after drag-n-drop operation (ref #123).
+  - Added drag-n-drop operation to move files in the project tree (ref #123).
+  - Added package AddMarker/RemoveMarker methods (ref #166).
+  - Added package GetStatusBar method (ref #166).
+  - Added package GetDocuments and document methods (ref #166).
+  - Added `EscapeMagic` function to escape magic characters.
+  - Added SciTeLuaIDE color scheme (thanks to Jayanth Acharya).
+  - Changed glslc option to reflect new version
+  - Disabled compilation check for scratchpad when `skipcompile` is set (ref #239).
+  - Disabled output activation for messages redirected to Console (ref #225).
+  - Disabled commenting for file types that don't specify line comments.
+  - Moved restoring project before loading files (ref #107).
+  - Reorganized loading configuration files (ref #107).
+  - Removed 'file no longer exists' message after moving opened files (ref #123).
+  - Removed some of the snippets as they are available as plugins.
+  - Store os specific clibs path to make it available to plugins.
+  - Tidy up estrela tools a bit (remove key strokes from cgc, remove luxinia res viewer, rename perforce files to get loaded again)
+  - Updated README.
+  - Updated samples with missing indicator constants (closes #243).
+  - Updated OSX executables and added 'fake' proxy for lua-dist support (ref #225).
+  - Updated handling of case-insensitive names during debugging on OSX (Mobdebug v0.545).
+  - Updated package onEditorPreSave to be called on SaveAs events (ref #166).
+  - Updated icon bundle to eliminate large icons.
+  - Updated application icon to a bit brighter one on OSX (closes #196).
+  - Updated build script on OSX to not require 10.6 SDK (closes #231).
+  - Updated menu definitions for consistency.
+  - Updated use of `unpack` for consistency and Lua 5.2 compatibility.
+  - Updated 'Open with Default Program' on Windows to work with spaces in names (#123).
+  - Updated cmake installation script to install to '/Applications' on OSX.
+  - Updated OSX build script to revert wxwidgets commit to fix auto-complete crash.
+  - Updated `Start debugging` hint to clarify.
+  - Updated single-click toggle to allow rename/move directories (ref #123).
+  - Updated normalization flags as some path parts were changed to dots.
+  - Updated editor tab processing using FindDocument method.
+  - Updated shortcut for Replace All to avoid conflict on OSX (fixes #220).
+  - Updated `SetDocumentModified` to use current tab text.
+
+### Fixes
+  - Fixed values 'captured' by redirected 'print' and not collected (fixes #240).
+  - Fixed typo in fold style definition (ref #237).
+  - Fixed console output being limited in addition to stack result limit.
+  - Fixed hang in auto-complete on expressions involving '...' (fixes #235).
+  - Fixed auto-complete for string values (broken by 933aacc2).
+  - Fixed crash when LUA_DEV environmental variable is not set (fixes #228).
+  - Fixed cmake cache filename (ref #225).
+  - Fixed incorrect UTF-8 sequence in UTF-8 validity check.
+  - Fixed Un/Comment menu for Output/Console windows.
+  - Fixed format of the file name reported after compilation errors.
+  - Fixed jumping to compilation error (if any) after Run/Debug.
+  - Fixed disabling 'Open with Default Program' on OSX (ref #123).
+  - Fixed file tree update after changes on OSX (ref #123).
+  - Fixed copying full path on OSX (ref #123).
+  - Fixed 'Open with Default Program' for paths with spaces on Windows (ref #123).
+  - Fixed folding issue (caused by `math.mod` not available in LuaJIT).
+  - Fixed debugger marker calculations to avoid marker conflicts.
+  - Fixed color references in marker config examples.
+  - Fixed Step Over/Out to stay in the same coroutine; Mobdebug 0.543 (closes #217).
+  - Fixed case sensitivity in matching of file name in error messages (fixes #216).
+  - Fixed tab text after SaveAs and loading files into the same tab.
+
+## v0.39 (Oct 06 2013)
+
+### Highlights
+  - Added Lua 5.2 support out of the box.
+  - Added suggesting dynamic words as fields in auto-complete.
+  - Added 'go to definition' (Ctrl/Cmd+Alt+Click) and 'jump to previous location' (Alt+Left).
+  - Added abbreviation of project directories to keep unique parts visible.
+  - Fixed breakpoints with Marmalade Quick.
+  - Switched to using LuaJIT interpreter by default.
+  - Upgraded Luasocket (3.0-rc1), copas, and coxpcall libraries.
 
 ### Special thanks
   - To Chow CheeWen for Chinese translation.
+  - To [Enrique García](https://github.com/kikito) for fixing `fixutf8` function.
   - To [Riidom](https://github.com/Riidom) for German translation.
-  - To [ardente](https://github.com/ardente) for user home patch for Windows.
+  - To [ardente](https://github.com/ardente) for user home patch for Windows and separating Lua 5.1 and 5.2 paths in user config.
   - To [Mika Attila](https://github.com/crumblingstatue) for code folding patch.
   - To [Tim Mensch](https://github.com/TimMensch) for auto-save, auto-reload, and debugger improvements.
   - To [Florian](https://github.com/SiENcE) for Notepad++ color scheme.
+  - To [Michal Kottman](https://github.com/mkottman) for 'go to definition' and Alt+Left navigation patch.
+  - To [Christoph Kubisch](https://github.com/CrazyButcher) for dx11 and glslc updates.
+  - To [jpoag](https://github.com/jpoag) for improved activation during debugging on Windows.
 
 ### Improvements
+  - Added setting project directory when passed as a parameter.
+  - Added activation of windows with SDL_app class name (Moai support).
+  - Added support for joining/splitting Watch/Stack with Output/Console tabs.
+  - Added package GetSetting method (ref #166).
+  - Added selected index to package onMenuEditorTab event (ref #166).
+  - Added activation of windows with FREEGLUT class name (Moai support).
+  - Added hiding console window for Corona (2013.8.28+) applications (Windows).
+  - Added suggesting dynamic words as fields in auto-complete.
+  - Added socket.connect for compatibility with socket.core <3.0 (fixes #208).
+  - Added recalculating line number margin width after zooming (fixes #207).
+  - Added margin constants and removed unused variables.
+  - Added reporting of socket error for initial debugger calls (Mobdebug 0.5403).
+  - Added error handling/reporting for `debugger.outputfilter`.
+  - Added 'debug' option to OSX build script.
+  - Added ability to modify exe path in base interpreter (ref #197).
+  - Added package GetEditorNotebook method (ref #166).
+  - Added 'molokai' color scheme (ref #200).
+  - added hlsl spec and basic api (note: currently autocomplete doesn't work on object functions, need fix). also fixed cg syntax lexer settings
+  - Added file activation for abbreviated file names in error messages.
+  - Added abbreviation of project directories to keep unique parts visible.
+  - Added `debugger.redirect` configuration option.
+  - Added `editor.saveallonrun` configuration option.
   - Added package GetOutput method (ref #166).
   - Added package onAppLoad/onAppClose events (ref #166).
   - Added package onIdleOnce event (ref #166).
@@ -47,28 +195,50 @@
   - Added an example of styling individual keywords.
   - Added fold indication of a current block (ref #168).
   - Added reporting of process id for a conflicting process.
+  - allow to define separate lua 5.1 and 5.2 paths in user config
+  - Changed 'go to definition' to Ctrl/Cmd+Alt+Click (ref #203).
   - Changed `un/comment` to act from the beginning of the line for multi-line selection.
+  - Disabled refreshing Watch/Stack windows when they get focus.
   - Disabled markup styling for specs without comment styles.
   - Disabled showing tooltip when auto-complete suggestions are shown.
   - Disabled error reporting after debugging has been terminated.
   - Disabled 'Fold' menu instead of removing when no folding is allowed (ref #169).
+  - dx11 and glslc updates
+  - Enabled editing watches with doubleclick or Enter.
+  - Enable Ctrl+Click and Alt+Left navigation on local variables
   - Enabled support for xml/html folding.
   - Enabled path remapping for local debugging.
   - Enabled slower and more thorough static analysis (ref #149; ref #168).
+  - Improved file/debugger activation on Windows (ref #199).
+  - Improved IDE activation during debugging on Windows (closes #199); thanks to jpoag.
   - Improved logic to jump to file/line indicated in error messages.
   - Limited activation of code fragments to the beginning of debugging session.
   - Make code folding optional (thanks to [Mika Attila](https://github.com/crumblingstatue))
   - Moved 'Sort' menu to 'Edit'.
   - OpenGL 4.4 and ARB extensions added as well as NV_gpu_shader5 functions
+  - Optimized handling of large tables in stack results.
   - Optimized line count calculation for dynamic words when text is deleted.
   - Optimized dynamic word processing for large files.
+  - Reduced CPU usage while idle (ref #204, #206).
   - Renamed package onEditorPostSave event to onEditorSave for consistency (ref #166).
   - Removed comment from default spec as it forces undesired markup styling.
   - Removed auto-complete suggestion when it is already typed (ref #101).
+  - Reorganized auto-complete handling; should fix #164.
+  - Reorganized path separator handling to minimize use of global variables.
   - Reorganized API processing to allow loading API description from a plugin.
   - Replaced package onEditorActivated event with onEditorFocus* events (ref #166).
   - Set search in subdirectories as default in Find in Files dialog (ref #162).
   - Switched to using POSIX compatible regexp with '()' used for captures.
+  - Updated LICENSE information.
+  - Updated Windows build file for wxwidgets 3.0.
+  - Updated support for MOAI coroutine debugging (Mobdebug 0.541).
+  - Updated type assignment logic to remove spurious types.
+  - Updated Windows build script to enable gdb debugging.
+  - Updated OSX build script to not strip debug builds.
+  - Updated Corona interpreter to handle failure to copy debugger to Resources/ folder.
+  - Updated build scripts with proper INSTALL_PREFIX option.
+  - Updated CFBundleIdentifier in plist files to allow references from OSX programs.
+  - Updated un/comment to toggle selection as a group rather than line by line.
   - Updated `NewFile` to accept a file name.
   - Updated 'get hostname' logic to avoid using non-resolvable names (mostly on OSX).
   - Updated tooltip to use the same type inference as auto-complete (ref #101).
@@ -79,9 +249,25 @@
   - Updated static analyzer to report only first instance of 'unknown field'.
   - Updated filename/source code heuristic in the debugger (Mobdebug 0.5362).
   - Updated `SaveAll` to allow saving (only) files with filenames (ref #172).
+  - Upgraded copas and coxpcall libraries (closes #144).
   - windows: adopt native user home (thanks to [ardente](https://github.com/ardente))
 
 ### Fixes
+  - Fixed Lua 5.2 crash on OSX (added -O1).
+  - Fixed onInterpreterLoad to only be called when interpreter changes.
+  - fix the fixutf8 function (thanks to Enrique García).
+  - Fixed handling of source code fragments in the Stack view.
+  - Fixed Watch/Stack windows to refresh when shown.
+  - Fixed incorrect editor tab acted upon in split notebook situations.
+  - Fixed auto-complete suggestions for indentifiers matching partial function names.
+  - Fixed hiding launched windows when running/debugging (Windows).
+  - Fixed showing known functions in auto-complete.
+  - Fixed showing output with invalid UTF8 characters in Stack and Console windows.
+  - Fixed debugging on/off handling in 'main' thread for LuaJIT (MobDebug 0.5402).
+  - Fixed having duplicate tabs after SaveAs with existing file name.
+  - Fixed showing redirected 'print' messages after debugging is terminated.
+  - Fixed using default interpreter when no interpreter is selected.
+  - Fixed stepping through blocks with undefined variables when 'strict' is in effect (upgraded Mobdebug to 0.5401).
   - Fixed loading of files with incorrect UTF-8 encoding and control characters (fixes #198).
   - Fixed package sample to take into account new documents.
   - Fixed crash on OSX after opening 'application' in 'Open File' dialog.
@@ -530,7 +716,7 @@
 ### Special thanks
   - To Andy Bower and Atilim Cetin for their assistance with Gideros integration and live coding.
   - To toiffel for Linux/OSX/Windows CMake-based launcher build.
-  - To Christoph Kubisch for help with Estrela merge.
+  - To [Christoph Kubisch](https://github.com/CrazyButcher) for help with Estrela merge.
 
 ### Improvements
   - Added configuration option to specify hostname when the default one is not reachable (fixes #68).

LICENSE

@@ -136,43 +136,59 @@ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 
 SOFTWARE. 
 
---[[ LuaFileSystem License ]]----------------------------------------------
-
-LuaFileSystem was designed by Roberto Ierusalimschy, André Carregal and Tomás 
-Guisasola as part of the Kepler Project, which holds its copyright.
-Homepage: http://www.keplerproject.org/luafilesystem/index.html
-License:  http://www.keplerproject.org/luafilesystem/license.html
-
 --[[ LuaSockets License ]]-------------------------------------------------
 
 Copyright: © 2004-2006 Diego Nehab. All rights reserved.
 Homepage: http://www.cs.princeton.edu/~diego/professional/luasocket/
 License:  http://www.lua.org/copyright.html (same as LUA)
 
---[[ ZMQ License ]]--------------------------------------------------------
+--[[ Serpent License ]]----------------------------------------------------
 
-Copyright: © 2007-2011 iMatix Corporation and Contributors
-Homepage: http://www.zeromq.org/
-License:  http://www.zeromq.org/area:licensing
+Copyright (c) 2011-2013 Paul Kulchenko (paul@kulchenko.com)
 
-LuaZMQ
-Copyright: © 2011 by Robert G. Jakabosky
-Homepage: https://github.com/Neopallium/lua-zmq
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
 
---[[ MojoShader License ]]-------------------------------------------------
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
 
-Copyright: © 2008-2010 Ryan C. Gordon
-Homepage: http://icculus.org/mojoshader/
-License:  http://hg.icculus.org/icculus/mojoshader/raw-file/tip/LICENSE.txt
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.
 
---[[ CLCC License ]]-------------------------------------------------------
+--[[ LuaJIT License ]]-----------------------------------------------------
 
-Copyright: © 2009 Organic Vectory B.V.
-Homepage: http://clcc.sourceforge.net/
-License:  boost
+Copyright © 2005-2013 Mike Pall, released under the MIT open source license.
 
-CLCC was modified by Christoph Kubisch to support multiple platforms and
-output file generation.
+--[[ WinAPI License ]]-----------------------------------------------------
+
+Copyright (C) 2011 Steve Donovan.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.
 
 --[[ MobDebug License ]]---------------------------------------------------
 

README.md

@@ -11,20 +11,29 @@ MobileLua, GSL-shell, and others). It originated from the [Estrela Editor](http:
 * Small, portable, and cross-platform (Windows, Mac OSX, and Linux).
 * Auto-completion for functions, keywords, and custom APIs.
 * Interactive console to directly test code snippets with local and remote execution.
-* Integrated debugger (with support for local and remote debugging).
-* Live coding with Lua ([demo](http://notebook.kulchenko.com/zerobrane/live-coding-in-lua-bret-victor-style)), Löve 2D ([demo](http://notebook.kulchenko.com/zerobrane/live-coding-with-love)), Gideros ([demo](http://notebook.kulchenko.com/zerobrane/gideros-live-coding-with-zerobrane-studio-ide)), Moai ([demo](http://notebook.kulchenko.com/zerobrane/live-coding-with-moai-and-zerobrane-studio)), and Corona SDK ([demo](http://notebook.kulchenko.com/zerobrane/debugging-and-live-coding-with-corona-sdk-applications-and-zerobrane-studio)).
+* Integrated debugger with local and [remote debugging](http://studio.zerobrane.com/doc-remote-debugging.html) for Lua 5.1,
+[Lua 5.2](http://studio.zerobrane.com/doc-lua52-debugging.html),
+[LuaJIT](http://studio.zerobrane.com/doc-luajit-debugging.html),
+and [other Lua engines](http://studio.zerobrane.com/documentation.html#debugging).
+* Live coding with Lua ([demo](http://notebook.kulchenko.com/zerobrane/live-coding-in-lua-bret-victor-style)),
+Löve 2D ([demo](http://notebook.kulchenko.com/zerobrane/live-coding-with-love)),
+Gideros ([demo](http://notebook.kulchenko.com/zerobrane/gideros-live-coding-with-zerobrane-studio-ide)),
+Moai ([demo](http://notebook.kulchenko.com/zerobrane/live-coding-with-moai-and-zerobrane-studio)),
+and Corona SDK ([demo](http://notebook.kulchenko.com/zerobrane/debugging-and-live-coding-with-corona-sdk-applications-and-zerobrane-studio)).
 * Support for plugin-like components:
-  - specs (spec/): file syntax, lexer, keywords (e.g. glsl);
-  - apis (api/): for code-completion and tool-tips;
-  - interpreters (interpreters/): how a project is run;
-  - config (cfg/): contains style and basic editor settings;
-  - tools (tools/): additional tools (e.g. DirectX/Cg shader compiler...).
+  - specs (`spec/`): file syntax, lexer, and keywords;
+  - apis (`api/`): for code completion and tooltips;
+  - interpreters (`interpreters/`): components for setting debugging and run-time project environment;
+  - config (`cfg/`): contains style, color themes, and other settings;
+  - packages (`packages/`): plugins that provide additional functionality;
+  - tools (`tools/`): additional tools.
 
 ## Documentation
 
 * A [short and simple overview](http://studio.zerobrane.com/doc-getting-started.html) for those who are new to this development environment.
 * A list of [frequently asked questions](http://studio.zerobrane.com/doc-faq.html) about the IDE.
 * [Tutorials and demos](http://studio.zerobrane.com/tutorials.html) that cover debugging and live coding for different environments.
+* [Tips and tricks](http://studio.zerobrane.com/doc-tips-and-tricks.html).
 
 ## Screenshot
 
@@ -37,6 +46,10 @@ Open file(s):
   zbstudio <filename> [<filename>...]
   any non-option will be treated as filename
 
+Set project directory:
+  zbstudio <project directory> [<filename>...]
+  (0.39+) a directory passed as a parameter will be set as the project directory
+
 Overriding default configuration:
   zbstudio -cfg "<luacode overriding config>" [<filename>]
   e.g.: zbstudio -cfg "editor.fontsize=12" somefile.lua

api/glsl/std.lua

@@ -234,6 +234,7 @@ local keyw =
     gl_NumWorkGroups gl_WorkGroupSize gl_WorkGroupID gl_LocalInvocationID gl_GlobalInvocationID gl_LocalInvocationIndex
     local_size_x local_size_y local_size_z
     gl_BaseVertexARB gl_BaseInstanceARB gl_DrawIDARB
+    bindless_sampler bound_sampler bindless_image bound_image
 
     coherent volatile restrict readonly writeonly
     image1D image2D image3D image2DRect imageCube imageBuffer image1DArray image2DArray imageCubeArray image2DMS image2DMSArray

api/hlsl/dx11.lua

@@ -0,0 +1,60 @@
+-- authors: Luxinia Dev (Eike Decker & Christoph Kubisch)
+---------------------------------------------------------
+
+local function fn (description) 
+	local description2,returns,args = description:match("(.+)%-%s*(%b())%s*(%b())")
+	if not description2 then
+		return {type="function",description=description,
+			returns="(?)"} 
+	end
+	return {type="function",description=description2,
+		returns=returns:gsub("^%s+",""):gsub("%s+$",""), args = args} 
+end
+
+local api = {}
+
+local funcs = [[
+abs acos all AllMemoryBarrier AllMemoryBarrierWithGroupSync any asdouble asfloat asin asint asuint atan atan2 ceil clamp clip cos cosh countbits cross ddx ddx_coarse ddx_fine ddy ddy_coards ddy_fine degrees determinant DeviceMemoryBarrier DeviceMemoryBarrierWithGroupSync distance dot dst EvaluateAttributeAtCentroid EvaluateAttributeAtSample EvaluateAttributeSnapped exp exp2 f16tof32 f32tof16 faceforward firstbithigh firstbitlow floor fmod frac frexp fwidth GetRenderTargetSampleCount GetRenderTargetSamplePosition GroupMemoryBarrier GroupMemoryBarrierWithGroupSync InterlockedAdd InterlockedAnd InterlockedCompareExchange InterlockedExchange InterlockedMax InterlockedMin IntterlockedOr InterlockedXor isfinite isinf isnan ldexp length lerp lit log log10 log2 mad max min modf mul normalize pow Process2DQuadTessFactorsAvg Process2DQuadTessFactorsMax Process2DQuadTessFactorsMin ProcessIsolineTessFactors ProcessQuadTessFactorsAvg ProcessQuadTessFactorsMax ProcessQuadTessFactorsMin ProcessTriTessFactorsAvg ProcessTriTessFactorsMax ProcessTriTessFactorsMin radians rcp reflect refract reversebits round rsqrt saturate sign sin sincos sinh smoothstep sqrt step tan tanh transpose trunc
+]]
+
+for w in funcs:gmatch("([_%w]+)") do
+	api[w] = {type="function",returns="(?)"}
+end
+
+local objfuncs = [[
+Append RestartStrip CalculateLevelOfDetail CalculateLevelOfDetailUnclamped GetDimensions GetSamplePosition Load Sample SampleBias SampleCmp SampleCmpLevelZero SampleGrad SampleLevel Load2 Load3 Load4 Consume Store Store2 Store3 Store4 DecrementCounter IncrementCounter mips Gather GatherRed GatherGreen GatherBlue GatherAlpha GatherCmp GatherCmpRed GatherCmpGreen GatherCmpBlue GatherCmpAlpha
+]]
+
+for w in objfuncs:gmatch("([_%w]+)") do
+	api[w] = {type="function",returns="(?)"}
+end
+
+local keyw = 
+[[break continue if else switch return for while do typedef namespace true false compile
+    const void struct static extern register volatile inline target nointerpolation shared uniform row_major column_major snorm unorm 
+    bool bool1 bool2 bool3 bool4 int int1 int2 int3 int4 uint uint1 uint2 uint3 uint4 half half1 half2 half3 half4 float float1 float2 float3 float4 double double1 double2 double3 double4
+    matrix bool1x1 bool1x2 bool1x3 bool1x4 bool2x1 bool2x2 bool2x3 bool2x4 bool3x1 bool3x2 bool3x3 bool3x4 bool4x1 bool4x2 bool4x3 bool4x4
+    int1x1 int1x2 int1x3 int1x4 int2x1 int2x2 int2x3 int2x4 int3x1 int3x2 int3x3 int3x4 int4x1 int4x2 int4x3 int4x4 uint1x1 uint1x2 uint1x3 uint1x4 
+    uint2x1 uint2x2 uint2x3 uint2x4 uint3x1 uint3x2 uint3x3 uint3x4 uint4x1 uint4x2 uint4x3 uint4x4 half1x1 half1x2 half1x3 half1x4 half2x1 half2x2 
+    half2x3 half2x4 half3x1 half3x2 half3x3 half3x4 half4x1 half4x2 half4x3 half4x4 float1x1 float1x2 float1x3 float1x4 float2x1 float2x2 float2x3
+    float2x4 float3x1 float3x2 float3x3 float3x4 float4x1 float4x2 float4x3 float4x4 double1x1 double1x2 double1x3 double1x4 double2x1 double2x2 
+    double2x3 double2x4 double3x1 double3x2 double3x3 double3x4 double4x1 double4x2 double4x3 double4x4 cbuffer groupshared SamplerState 
+    in out inout vector matrix interface class point triangle line lineadj triangleadj
+    
+    Texture Texture1D Texture1DArray Texture2D Texture2DArray Texture2DMS Texture2DMSArray Texture3D TextureCube RWTexture1D RWTexture1DArray RWTexture2D RWTexture2DArray RWTexture3D 
+    Buffer StructuredBuffer AppendStructuredBuffer ConsumeStructuredBuffer RWBuffer RWStructuredBuffer ByteAddressBuffer RWByteAddressBuffer PointStream TriangleStream LineStream InputPatch OutputPatch
+    unroll loop flatten branch earlydepthstencil allow_uav_condition domain instance maxtessfactor outputcontrolpoints outputtopology partitioning patchconstantfunc numthreads maxvertexcount precise
+    
+    SV_DispatchThreadID SV_DomainLocation SV_GroupID SV_GroupIndex SV_GroupThreadID SV_GSInstanceID SV_InsideTessFactor SV_OutputControlPointID SV_Coverage SV_Depth SV_Position SV_IsFrontFace SV_RenderTargetArrayIndex SV_SampleIndex SV_ViewportArrayIndex SV_InstanceID SV_PrimitiveID SV_VertexID    
+    SV_ClipDistance SV_CullDistance SV_Target
+
+]]
+
+-- keywords - shouldn't be left out
+for w in keyw:gmatch("([_%w]+)") do
+	api[w] = {type="keyword"}
+end
+
+return api
+
+

bin/lua

@@ -0,0 +1 @@
+lua.app/Contents/MacOS/lua

bin/lua52

@@ -0,0 +1 @@
+lua.app/Contents/MacOS/lua52

build/CMakeLists.txt

@@ -84,6 +84,8 @@ if(WIN32)
   install_from_manifest(FILES ${TOPDIR}/zbstudio/MANIFEST ${DATADIR} "^$")
   install_from_manifest(FILES ${TOPDIR}/zbstudio/MANIFEST-bin-win32 ${DATADIR} "^zbstudio.exe$")
 elseif(APPLE)
+  set(OSX_INSTALL_PREFIX "/Applications" CACHE PATH "Default OSX prefix")
+  set(CMAKE_INSTALL_PREFIX "${OSX_INSTALL_PREFIX}" CACHE INTERNAL "Prefix prepended to install directories")
   # setup the data directory
   set(ROOTDIR ZeroBraneStudio.app/Contents)
   set(DATADIR ${ROOTDIR}/ZeroBraneStudio)

build/build-linux.sh

@@ -32,16 +32,12 @@ ZLIB_BASENAME="zlib-1.2.8"
 ZLIB_FILENAME="$ZLIB_BASENAME.tar.gz"
 ZLIB_URL="https://github.com/madler/zlib/archive/v1.2.8.tar.gz"
 
-LUA_BASENAME="lua-5.1.5"
-LUA_FILENAME="$LUA_BASENAME.tar.gz"
-LUA_URL="http://www.lua.org/ftp/$LUA_FILENAME"
-
 WXLUA_BASENAME="wxlua"
-WXLUA_URL="https://wxlua.svn.sourceforge.net/svnroot/wxlua/trunk"
+WXLUA_URL="https://svn.code.sf.net/p/wxlua/svn/trunk"
 
-LUASOCKET_BASENAME="luasocket-2.0.3"
-LUASOCKET_FILENAME="$LUASOCKET_BASENAME-rc2.zip"
-LUASOCKET_URL="https://github.com/downloads/diegonehab/luasocket/$LUASOCKET_FILENAME"
+LUASOCKET_BASENAME="luasocket-3.0-rc1"
+LUASOCKET_FILENAME="v3.0-rc1.zip"
+LUASOCKET_URL="https://github.com/diegonehab/luasocket/archive/$LUASOCKET_FILENAME"
 
 # exit if the command line is empty
 if [ $# -eq 0 ]; then
@@ -52,6 +48,12 @@ fi
 # iterate through the command line arguments
 for ARG in "$@"; do
   case $ARG in
+  5.2)
+    BUILD_52=true
+    ;;
+  jit)
+    BUILD_JIT=true
+    ;;
   wxwidgets)
     BUILD_WXWIDGETS=true
     ;;
@@ -104,6 +106,25 @@ fi
 # create the installation directory
 mkdir -p "$INSTALL_DIR" || { echo "Error: cannot create directory $INSTALL_DIR"; exit 1; }
 
+LUAV="51"
+LUAS=""
+LUA_BASENAME="lua-5.1.5"
+
+if [ $BUILD_52 ]; then
+  LUAV="52"
+  LUAS=$LUAV
+  LUA_BASENAME="lua-5.2.2"
+fi
+
+LUA_FILENAME="$LUA_BASENAME.tar.gz"
+LUA_URL="http://www.lua.org/ftp/$LUA_FILENAME"
+
+if [ $BUILD_JIT ]; then
+  LUA_BASENAME="LuaJIT-2.0.2"
+  LUA_FILENAME="$LUA_BASENAME.tar.gz"
+  LUA_URL="http://luajit.org/download/$LUA_FILENAME"
+fi
+
 # build wxWidgets
 if [ $BUILD_WXWIDGETS ]; then
   # first build get/configure libpng as v1.6 is needed
@@ -142,10 +163,21 @@ if [ $BUILD_LUA ]; then
   wget -c "$LUA_URL" -O "$LUA_FILENAME" || { echo "Error: failed to download Lua"; exit 1; }
   tar -xzf "$LUA_FILENAME"
   cd "$LUA_BASENAME"
-  # use POSIX as it has minimum dependencies (no readline and no ncurses required)
-  # LUA_USE_DLOPEN is required for loading libraries
-  (cd src; make all MYCFLAGS="$FPIC -DLUA_USE_POSIX -DLUA_USE_DLOPEN" MYLIBS="-Wl,-E -ldl") || { echo "Error: failed to build Lua"; exit 1; }
-  make install INSTALL_TOP="$INSTALL_DIR"
+
+  if [ $BUILD_JIT ]; then
+    make CCOPT="-DLUAJIT_ENABLE_LUA52COMPAT" || { echo "Error: failed to build Lua"; exit 1; }
+    make install PREFIX="$INSTALL_DIR"
+    cp "$INSTALL_DIR/bin/luajit" "$INSTALL_DIR/bin/lua"
+    # move luajit to lua as it's expected by luasocket and other components
+    cp "$INSTALL_DIR"/include/luajit*/* "$INSTALL_DIR/include/"
+  else
+    # use POSIX as it has minimum dependencies (no readline and no ncurses required)
+    # LUA_USE_DLOPEN is required for loading libraries
+    (cd src; make all MYCFLAGS="$FPIC -DLUA_USE_POSIX -DLUA_USE_DLOPEN" MYLIBS="-Wl,-E -ldl") || { echo "Error: failed to build Lua"; exit 1; }
+    make install INSTALL_TOP="$INSTALL_DIR"
+  fi
+  cp "$INSTALL_DIR/bin/lua" "$INSTALL_DIR/bin/lua$LUAV"
+
   cd ..
   rm -rf "$LUA_FILENAME" "$LUA_BASENAME"
 fi
@@ -157,7 +189,7 @@ if [ $BUILD_WXLUA ]; then
   # the following patches wxlua source to fix live coding support in wxlua apps
   # http://www.mail-archive.com/wxlua-users@lists.sourceforge.net/msg03225.html
   sed -i 's/\(m_wxlState = wxLuaState(wxlState.GetLuaState(), wxLUASTATE_GETSTATE|wxLUASTATE_ROOTSTATE);\)/\/\/ removed by ZBS build process \/\/ \1/' modules/wxlua/wxlcallb.cpp
-  cmake -G "Unix Makefiles" -DBUILD_INSTALL_PREFIX="$INSTALL_DIR" -DCMAKE_BUILD_TYPE=MinSizeRel -DBUILD_SHARED_LIBS=FALSE \
+  cmake -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX="$INSTALL_DIR" -DCMAKE_BUILD_TYPE=MinSizeRel -DBUILD_SHARED_LIBS=FALSE \
     -DwxWidgets_CONFIG_EXECUTABLE="$INSTALL_DIR/bin/wx-config" \
     -DwxWidgets_COMPONENTS="stc;html;aui;adv;core;net;base" \
     -DwxLuaBind_COMPONENTS="stc;html;aui;adv;core;net;base" -DwxLua_LUA_LIBRARY_USE_BUILTIN=FALSE \
@@ -174,31 +206,30 @@ if [ $BUILD_LUASOCKET ]; then
   wget --no-check-certificate -c "$LUASOCKET_URL" -O "$LUASOCKET_FILENAME" || { echo "Error: failed to download LuaSocket"; exit 1; }
   unzip "$LUASOCKET_FILENAME"
   cd "$LUASOCKET_BASENAME"
-  mkdir -p "$INSTALL_DIR/lib/lua/5.1/"{mime,socket}
-  gcc $BUILD_FLAGS -o "$INSTALL_DIR/lib/lua/5.1/mime/core.so" src/mime.c -llua \
+  mkdir -p "$INSTALL_DIR/lib/lua/$LUAV/"{mime,socket}
+  gcc $BUILD_FLAGS -o "$INSTALL_DIR/lib/lua/$LUAV/mime/core.so" src/mime.c -llua \
     || { echo "Error: failed to build LuaSocket"; exit 1; }
-  gcc $BUILD_FLAGS -o "$INSTALL_DIR/lib/lua/5.1/socket/core.so" \
+  gcc $BUILD_FLAGS -o "$INSTALL_DIR/lib/lua/$LUAV/socket/core.so" \
     src/{auxiliar.c,buffer.c,except.c,inet.c,io.c,luasocket.c,options.c,select.c,tcp.c,timeout.c,udp.c,usocket.c} -llua \
     || { echo "Error: failed to build LuaSocket"; exit 1; }
-  mkdir -p "$INSTALL_DIR/share/lua/5.1/socket"
-  cp src/{ftp.lua,http.lua,smtp.lua,tp.lua,url.lua} "$INSTALL_DIR/share/lua/5.1/socket"
-  cp src/{ltn12.lua,mime.lua,socket.lua} "$INSTALL_DIR/share/lua/5.1"
-  [ -f "$INSTALL_DIR/lib/lua/5.1/mime/core.so" ] || { echo "Error: mime/core.so isn't found"; exit 1; }
-  [ -f "$INSTALL_DIR/lib/lua/5.1/socket/core.so" ] || { echo "Error: socket/core.so isn't found"; exit 1; }
+  mkdir -p "$INSTALL_DIR/share/lua/$LUAV/socket"
+  cp src/{ftp.lua,http.lua,smtp.lua,tp.lua,url.lua} "$INSTALL_DIR/share/lua/$LUAV/socket"
+  cp src/{ltn12.lua,mime.lua,socket.lua} "$INSTALL_DIR/share/lua/$LUAV"
+  [ -f "$INSTALL_DIR/lib/lua/$LUAV/mime/core.so" ] || { echo "Error: mime/core.so isn't found"; exit 1; }
+  [ -f "$INSTALL_DIR/lib/lua/$LUAV/socket/core.so" ] || { echo "Error: socket/core.so isn't found"; exit 1; }
   cd ..
   rm -rf "$LUASOCKET_FILENAME" "$LUASOCKET_BASENAME"
 fi
 
 # now copy the compiled dependencies to ZBS binary directory
 mkdir -p "$BIN_DIR" || { echo "Error: cannot create directory $BIN_DIR"; exit 1; }
-[ $BUILD_LUA ] && cp "$INSTALL_DIR/bin/lua" "$BIN_DIR"
+[ $BUILD_LUA ] && cp "$INSTALL_DIR/bin/lua$LUAS" "$BIN_DIR"
 [ $BUILD_WXLUA ] && cp "$INSTALL_DIR/lib/libwx.so" "$BIN_DIR"
 if [ $BUILD_LUASOCKET ]; then
-  mkdir -p "$BIN_DIR/clibs/"{mime,socket}
-  cp "$INSTALL_DIR/lib/lua/5.1/mime/core.so" "$BIN_DIR/clibs/mime"
-  cp "$INSTALL_DIR/lib/lua/5.1/socket/core.so" "$BIN_DIR/clibs/socket"
+  mkdir -p "$BIN_DIR/clibs$LUAS/"{mime,socket}
+  cp "$INSTALL_DIR/lib/lua/$LUAV/mime/core.so" "$BIN_DIR/clibs$LUAS/mime"
+  cp "$INSTALL_DIR/lib/lua/$LUAV/socket/core.so" "$BIN_DIR/clibs$LUAS/socket"
 fi
 
-# show a message about successful completion
 echo "*** Build has been successfully completed ***"
 exit 0

build/build-macosx.sh

@@ -15,23 +15,23 @@ MACOSX_SDK_PATH="/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.pla
 MAKEFLAGS="-j4"
 
 # flags for manual building with gcc; build universal binaries for luasocket
-MACOSX_FLAGS="-arch $MACOSX_ARCH -mmacosx-version-min=$MACOSX_VERSION -isysroot $MACOSX_SDK_PATH"
+MACOSX_FLAGS="-arch $MACOSX_ARCH -mmacosx-version-min=$MACOSX_VERSION"
+if [ -d "$MACOSX_SDK_PATH" ]; then
+  echo "Building with $MACOSX_SDK_PATH"
+  MACOSX_FLAGS="$MACOSX_FLAGS -isysroot $MACOSX_SDK_PATH"
+fi
 BUILD_FLAGS="-O2 -arch x86_64 -dynamiclib -undefined dynamic_lookup $MACOSX_FLAGS -I $INSTALL_DIR/include -L $INSTALL_DIR/lib"
 
 # paths configuration
 WXWIDGETS_BASENAME="wxWidgets"
 WXWIDGETS_URL="http://svn.wxwidgets.org/svn/wx/wxWidgets/trunk"
 
-LUA_BASENAME="lua-5.1.5"
-LUA_FILENAME="$LUA_BASENAME.tar.gz"
-LUA_URL="http://www.lua.org/ftp/$LUA_FILENAME"
-
 WXLUA_BASENAME="wxlua"
-WXLUA_URL="https://wxlua.svn.sourceforge.net/svnroot/wxlua/trunk"
+WXLUA_URL="https://svn.code.sf.net/p/wxlua/svn/trunk"
 
-LUASOCKET_BASENAME="luasocket-2.0.3"
-LUASOCKET_FILENAME="$LUASOCKET_BASENAME-rc2.zip"
-LUASOCKET_URL="https://github.com/downloads/diegonehab/luasocket/$LUASOCKET_FILENAME"
+LUASOCKET_BASENAME="luasocket-3.0-rc1"
+LUASOCKET_FILENAME="v3.0-rc1.zip"
+LUASOCKET_URL="https://github.com/diegonehab/luasocket/archive/$LUASOCKET_FILENAME"
 
 # exit if the command line is empty
 if [ $# -eq 0 ]; then
@@ -39,9 +39,19 @@ if [ $# -eq 0 ]; then
   exit 0
 fi
 
+WXLUASTRIP="/strip"
+WXWIDGETSDEBUG="--disable-debug"
+WXLUABUILD="MinSizeRel"
+
 # iterate through the command line arguments
 for ARG in "$@"; do
   case $ARG in
+  5.2)
+    BUILD_52=true
+    ;;
+  jit)
+    BUILD_JIT=true
+    ;;
   wxwidgets)
     BUILD_WXWIDGETS=true
     ;;
@@ -54,6 +64,11 @@ for ARG in "$@"; do
   luasocket)
     BUILD_LUASOCKET=true
     ;;
+  debug)
+    WXLUASTRIP=""
+    WXWIDGETSDEBUG="--enable-debug=max"
+    WXLUABUILD="Debug"
+    ;;
   all)
     BUILD_WXWIDGETS=true
     BUILD_LUA=true
@@ -94,14 +109,40 @@ fi
 # create the installation directory
 mkdir -p "$INSTALL_DIR" || { echo "Error: cannot create directory $INSTALL_DIR"; exit 1; }
 
+LUAV="51"
+LUAS=""
+LUA_BASENAME="lua-5.1.5"
+
+if [ $BUILD_52 ]; then
+  LUAV="52"
+  LUAS=$LUAV
+  LUA_BASENAME="lua-5.2.2"
+fi
+
+LUA_FILENAME="$LUA_BASENAME.tar.gz"
+LUA_URL="http://www.lua.org/ftp/$LUA_FILENAME"
+
+if [ $BUILD_JIT ]; then
+  LUA_BASENAME="LuaJIT-2.0.2"
+  LUA_FILENAME="$LUA_BASENAME.tar.gz"
+  LUA_URL="http://luajit.org/download/$LUA_FILENAME"
+fi
+
 # build wxWidgets
 if [ $BUILD_WXWIDGETS ]; then
   svn co "$WXWIDGETS_URL" "$WXWIDGETS_BASENAME" || { echo "Error: failed to checkout wxWidgets"; exit 1; }
   cd "$WXWIDGETS_BASENAME"
-  ./configure --prefix="$INSTALL_DIR" --disable-debug --disable-shared --enable-unicode \
+  MINSDK=""
+  if [ -d $MACOSX_SDK_PATH ]; then
+    MINSDK="--with-macosx-sdk=$MACOSX_SDK_PATH"
+  fi
+  # fix auto-complete crash by reverting an earlier revision
+  # http://trac.wxwidgets.org/ticket/15008#comment:12
+  svn merge -r 74098:74097 .
+  ./configure --prefix="$INSTALL_DIR" $WXWIDGETSDEBUG --disable-shared --enable-unicode \
     --with-libjpeg=builtin --with-libpng=builtin --with-libtiff=no --with-expat=no \
     --with-zlib=builtin --disable-richtext \
-    --enable-macosx_arch=$MACOSX_ARCH --with-macosx-version-min=$MACOSX_VERSION --with-macosx-sdk="$MACOSX_SDK_PATH" \
+    --enable-macosx_arch=$MACOSX_ARCH --with-macosx-version-min=$MACOSX_VERSION $MINSDK \
     --with-osx_cocoa CFLAGS="-Os" CXXFLAGS="-Os"
   make $MAKEFLAGS || { echo "Error: failed to build wxWidgets"; exit 1; }
   make install
@@ -114,16 +155,29 @@ if [ $BUILD_LUA ]; then
   wget -c "$LUA_URL" -O "$LUA_FILENAME" || { echo "Error: failed to download Lua"; exit 1; }
   tar -xzf "$LUA_FILENAME"
   cd "$LUA_BASENAME"
-  sed -i "" 's/PLATS=/& macosx_dylib/' Makefile
-  printf "macosx_dylib:\n" >> src/Makefile
-  printf "\t\$(MAKE) LUA_A=\"liblua.dylib\" AR=\"\$(CC) -dynamiclib $MACOSX_FLAGS -o\" RANLIB=\"strip -u -r\" \\\\\n" >> src/Makefile
-  printf "\tMYCFLAGS=\"-DLUA_USE_LINUX $MACOSX_FLAGS\" MYLDFLAGS=\"$MACOSX_FLAGS\" MYLIBS=\"-lreadline\" lua\n" >> src/Makefile
-  printf "\t\$(MAKE) MYCFLAGS=\"-DLUA_USE_LINUX $MACOSX_FLAGS\" MYLDFLAGS=\"$MACOSX_FLAGS\" luac\n" >> src/Makefile
-  make macosx_dylib || { echo "Error: failed to build Lua"; exit 1; }
-  make install INSTALL_TOP="$INSTALL_DIR"
-  strip -u -r "$INSTALL_DIR/bin/lua"
-  cp src/liblua.dylib "$INSTALL_DIR/lib"
-  [ -f "$INSTALL_DIR/lib/liblua.dylib" ] || { echo "Error: liblua.dylib isn't found"; exit 1; }
+
+  if [ $BUILD_JIT ]; then
+    make BUILDMODE=dynamic LUAJIT_SO=liblua.dylib TARGET_DYLIBPATH=liblua.dylib CC="gcc -m32" CCOPT="$MACOSX_FLAGS -DLUAJIT_ENABLE_LUA52COMPAT" || { echo "Error: failed to build Lua"; exit 1; }
+    make install PREFIX="$INSTALL_DIR"
+    cp "src/luajit" "$INSTALL_DIR/bin/lua"
+    cp "src/liblua.dylib" "$INSTALL_DIR/lib"
+    # move luajit to lua as it's expected by luasocket and other components
+    cp "$INSTALL_DIR"/include/luajit*/* "$INSTALL_DIR/include/"
+  else
+    sed -i "" 's/PLATS=/& macosx_dylib/' Makefile
+
+    # -O1 fixes this issue with for Lua 5.2 with i386: http://lua-users.org/lists/lua-l/2013-05/msg00070.html
+    printf "macosx_dylib:\n" >> src/Makefile
+    printf "\t\$(MAKE) LUA_A=\"liblua$LUAS.dylib\" AR=\"\$(CC) -dynamiclib $MACOSX_FLAGS -o\" RANLIB=\"strip -u -r\" \\\\\n" >> src/Makefile
+    printf "\tMYCFLAGS=\"-O1 -DLUA_USE_LINUX $MACOSX_FLAGS\" MYLDFLAGS=\"$MACOSX_FLAGS\" MYLIBS=\"-lreadline\" lua\n" >> src/Makefile
+    printf "\t\$(MAKE) MYCFLAGS=\"-DLUA_USE_LINUX $MACOSX_FLAGS\" MYLDFLAGS=\"$MACOSX_FLAGS\" luac\n" >> src/Makefile
+    make macosx_dylib || { echo "Error: failed to build Lua"; exit 1; }
+    make install INSTALL_TOP="$INSTALL_DIR"
+    mv "$INSTALL_DIR/bin/lua" "$INSTALL_DIR/bin/lua$LUAS"
+    cp src/liblua$LUAS.dylib "$INSTALL_DIR/lib"
+  fi
+  strip -u -r "$INSTALL_DIR/bin/lua$LUAS"
+  [ -f "$INSTALL_DIR/lib/liblua$LUAS.dylib" ] || { echo "Error: liblua$LUAS.dylib isn't found"; exit 1; }
   cd ..
   rm -rf "$LUA_FILENAME" "$LUA_BASENAME"
 fi
@@ -132,18 +186,22 @@ fi
 if [ $BUILD_WXLUA ]; then
   svn co "$WXLUA_URL" "$WXLUA_BASENAME" || { echo "Error: failed to checkout wxLua"; exit 1; }
   cd "$WXLUA_BASENAME/wxLua"
+  MINSDK=""
+  if [ -d $MACOSX_SDK_PATH ]; then
+    MINSDK="CMAKE_OSX_SYSROOT=$MACOSX_SDK_PATH"
+  fi
   # the following patches wxlua source to fix live coding support in wxlua apps
   # http://www.mail-archive.com/wxlua-users@lists.sourceforge.net/msg03225.html
   sed -i "" 's/\(m_wxlState = wxLuaState(wxlState.GetLuaState(), wxLUASTATE_GETSTATE|wxLUASTATE_ROOTSTATE);\)/\/\/ removed by ZBS build process \/\/ \1/' modules/wxlua/wxlcallb.cpp
-  cmake -G "Unix Makefiles" -DBUILD_INSTALL_PREFIX="$INSTALL_DIR" -DCMAKE_BUILD_TYPE=MinSizeRel -DBUILD_SHARED_LIBS=FALSE \
-    -DCMAKE_OSX_ARCHITECTURES=$MACOSX_ARCH -DCMAKE_OSX_DEPLOYMENT_TARGET=$MACOSX_VERSION CMAKE_OSX_SYSROOT="$MACOSX_SDK_PATH" \
+  cmake -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX="$INSTALL_DIR" -DCMAKE_BUILD_TYPE=$WXLUABUILD -DBUILD_SHARED_LIBS=FALSE \
+    -DCMAKE_OSX_ARCHITECTURES=$MACOSX_ARCH -DCMAKE_OSX_DEPLOYMENT_TARGET=$MACOSX_VERSION $MINSDK \
     -DCMAKE_C_COMPILER=/usr/bin/gcc -DCMAKE_CXX_COMPILER=/usr/bin/g++ -DwxWidgets_CONFIG_EXECUTABLE="$INSTALL_DIR/bin/wx-config" \
     -DwxWidgets_COMPONENTS="stc;html;aui;adv;core;net;base" \
     -DwxLuaBind_COMPONENTS="stc;html;aui;adv;core;net;base" -DwxLua_LUA_LIBRARY_USE_BUILTIN=FALSE \
     -DwxLua_LUA_INCLUDE_DIR="$INSTALL_DIR/include" -DwxLua_LUA_LIBRARY="$INSTALL_DIR/lib/liblua.dylib" .
   (cd modules/luamodule; make $MAKEFLAGS) || { echo "Error: failed to build wxLua"; exit 1; }
-  (cd modules/luamodule; make install/strip)
-  strip -u -r "$INSTALL_DIR/lib/libwx.dylib"
+  (cd modules/luamodule; make install$WXLUASTRIP)
+  if [ $WXLUASTRIP ]; then strip -u -r "$INSTALL_DIR/lib/libwx.dylib"; fi
   [ -f "$INSTALL_DIR/lib/libwx.dylib" ] || { echo "Error: libwx.dylib isn't found"; exit 1; }
   cd ../..
   rm -rf "$WXLUA_BASENAME"
@@ -154,36 +212,38 @@ if [ $BUILD_LUASOCKET ]; then
   wget --no-check-certificate -c "$LUASOCKET_URL" -O "$LUASOCKET_FILENAME" || { echo "Error: failed to download LuaSocket"; exit 1; }
   unzip "$LUASOCKET_FILENAME"
   cd "$LUASOCKET_BASENAME"
-  mkdir -p "$INSTALL_DIR/lib/lua/5.1/"{mime,socket}
-  gcc $BUILD_FLAGS -o "$INSTALL_DIR/lib/lua/5.1/mime/core.dylib" src/mime.c \
+  mkdir -p "$INSTALL_DIR/lib/lua/$LUAV/"{mime,socket}
+  gcc $BUILD_FLAGS -o "$INSTALL_DIR/lib/lua/$LUAV/mime/core.dylib" src/mime.c \
     || { echo "Error: failed to build LuaSocket"; exit 1; }
-  gcc $BUILD_FLAGS -o "$INSTALL_DIR/lib/lua/5.1/socket/core.dylib" \
+  gcc $BUILD_FLAGS -o "$INSTALL_DIR/lib/lua/$LUAV/socket/core.dylib" \
     src/{auxiliar.c,buffer.c,except.c,inet.c,io.c,luasocket.c,options.c,select.c,tcp.c,timeout.c,udp.c,usocket.c} \
     || { echo "Error: failed to build LuaSocket"; exit 1; }
-  strip -u -r "$INSTALL_DIR/lib/lua/5.1/mime/core.dylib" "$INSTALL_DIR/lib/lua/5.1/socket/core.dylib"
-  mkdir -p "$INSTALL_DIR/share/lua/5.1/socket"
-  cp src/{ftp.lua,http.lua,smtp.lua,tp.lua,url.lua} "$INSTALL_DIR/share/lua/5.1/socket"
-  cp src/{ltn12.lua,mime.lua,socket.lua} "$INSTALL_DIR/share/lua/5.1"
-  [ -f "$INSTALL_DIR/lib/lua/5.1/mime/core.dylib" ] || { echo "Error: mime/core.dylib isn't found"; exit 1; }
-  [ -f "$INSTALL_DIR/lib/lua/5.1/socket/core.dylib" ] || { echo "Error: socket/core.dylib isn't found"; exit 1; }
+  strip -u -r "$INSTALL_DIR/lib/lua/$LUAV/mime/core.dylib" "$INSTALL_DIR/lib/lua/$LUAV/socket/core.dylib"
+  install_name_tool -id core.dylib "$INSTALL_DIR/lib/lua/$LUAV/socket/core.dylib"
+  install_name_tool -id core.dylib "$INSTALL_DIR/lib/lua/$LUAV/mime/core.dylib"
+  mkdir -p "$INSTALL_DIR/share/lua/$LUAV/socket"
+  cp src/{ftp.lua,http.lua,smtp.lua,tp.lua,url.lua} "$INSTALL_DIR/share/lua/$LUAV/socket"
+  cp src/{ltn12.lua,mime.lua,socket.lua} "$INSTALL_DIR/share/lua/$LUAV"
+  [ -f "$INSTALL_DIR/lib/lua/$LUAV/mime/core.dylib" ] || { echo "Error: mime/core.dylib isn't found"; exit 1; }
+  [ -f "$INSTALL_DIR/lib/lua/$LUAV/socket/core.dylib" ] || { echo "Error: socket/core.dylib isn't found"; exit 1; }
   cd ..
   rm -rf "$LUASOCKET_FILENAME" "$LUASOCKET_BASENAME"
 fi
 
 # now copy the compiled dependencies to ZBS binary directory
 mkdir -p "$BIN_DIR" || { echo "Error: cannot create directory $BIN_DIR"; exit 1; }
+
 if [ $BUILD_LUA ]; then
   mkdir -p "$BIN_DIR/lua.app/Contents/MacOS"
-  cp "$INSTALL_DIR/bin/lua" "$BIN_DIR/lua.app/Contents/MacOS"
-  cp "$INSTALL_DIR/bin/lua" "$INSTALL_DIR/lib/liblua.dylib" "$BIN_DIR"
+  cp "$INSTALL_DIR/bin/lua$LUAS" "$BIN_DIR/lua.app/Contents/MacOS"
+  cp "$INSTALL_DIR/bin/lua$LUAS" "$INSTALL_DIR/lib/liblua$LUAS.dylib" "$BIN_DIR"
 fi
 [ $BUILD_WXLUA ] && cp "$INSTALL_DIR/lib/libwx.dylib" "$BIN_DIR"
 if [ $BUILD_LUASOCKET ]; then
-  mkdir -p "$BIN_DIR/clibs/"{mime,socket}
-  cp "$INSTALL_DIR/lib/lua/5.1/mime/core.dylib" "$BIN_DIR/clibs/mime"
-  cp "$INSTALL_DIR/lib/lua/5.1/socket/core.dylib" "$BIN_DIR/clibs/socket"
+  mkdir -p "$BIN_DIR/clibs$LUAS/"{mime,socket}
+  cp "$INSTALL_DIR/lib/lua/$LUAV/mime/core.dylib" "$BIN_DIR/clibs$LUAS/mime"
+  cp "$INSTALL_DIR/lib/lua/$LUAV/socket/core.dylib" "$BIN_DIR/clibs$LUAS/socket"
 fi
 
-# show a message about successful completion
 echo "*** Build has been successfully completed ***"
 exit 0

build/build-win32.sh

@@ -7,7 +7,7 @@ BIN_DIR="$(dirname "$PWD")/bin"
 INSTALL_DIR="$PWD/deps"
 
 # number of parallel jobs used for building
-MAKEFLAGS="-j4"
+MAKEFLAGS="-j1" # some make may hang on Windows with j4 or j7
 
 # flags for manual building with gcc
 BUILD_FLAGS="-O2 -shared -s -I $INSTALL_DIR/include -L $INSTALL_DIR/lib"
@@ -16,16 +16,12 @@ BUILD_FLAGS="-O2 -shared -s -I $INSTALL_DIR/include -L $INSTALL_DIR/lib"
 WXWIDGETS_BASENAME="wxWidgets"
 WXWIDGETS_URL="http://svn.wxwidgets.org/svn/wx/wxWidgets/trunk"
 
-LUA_BASENAME="lua-5.1.5"
-LUA_FILENAME="$LUA_BASENAME.tar.gz"
-LUA_URL="http://www.lua.org/ftp/$LUA_FILENAME"
-
 WXLUA_BASENAME="wxlua"
-WXLUA_URL="https://wxlua.svn.sourceforge.net/svnroot/wxlua/trunk"
+WXLUA_URL="https://svn.code.sf.net/p/wxlua/svn/trunk"
 
-LUASOCKET_BASENAME="luasocket-2.0.3"
-LUASOCKET_FILENAME="$LUASOCKET_BASENAME-rc2.zip"
-LUASOCKET_URL="https://github.com/downloads/diegonehab/luasocket/$LUASOCKET_FILENAME"
+LUASOCKET_BASENAME="luasocket-3.0-rc1"
+LUASOCKET_FILENAME="v3.0-rc1.zip"
+LUASOCKET_URL="https://github.com/diegonehab/luasocket/archive/$LUASOCKET_FILENAME"
 
 OPENSSL_BASENAME="openssl-1.0.1e"
 OPENSSL_FILENAME="$OPENSSL_BASENAME.tar.gz"
@@ -45,12 +41,18 @@ if [ $# -eq 0 ]; then
 fi
 
 WXLUASTRIP="/strip"
-WXLUABUILD="MinSizeRel"
 WXWIDGETSDEBUG="--disable-debug"
+WXLUABUILD="MinSizeRel"
 
 # iterate through the command line arguments
 for ARG in "$@"; do
   case $ARG in
+  5.2)
+    BUILD_52=true
+    ;;
+  jit)
+    BUILD_JIT=true
+    ;;
   wxwidgets)
     BUILD_WXWIDGETS=true
     ;;
@@ -74,7 +76,8 @@ for ARG in "$@"; do
     ;;
   debug)
     WXLUASTRIP=""
-    WXWIDGETSDEBUG="--enable-debug"
+    WXWIDGETSDEBUG="--enable-debug=max --enable-debug_gdb"
+    WXLUABUILD="Debug"
     ;;
   all)
     BUILD_WXWIDGETS=true
@@ -124,6 +127,25 @@ fi
 # create the installation directory
 mkdir -p "$INSTALL_DIR" || { echo "Error: cannot create directory $INSTALL_DIR"; exit 1; }
 
+LUAV="51"
+LUAS=""
+LUA_BASENAME="lua-5.1.5"
+
+if [ $BUILD_52 ]; then
+  LUAV="52"
+  LUAS=$LUAV
+  LUA_BASENAME="lua-5.2.2"
+fi
+
+LUA_FILENAME="$LUA_BASENAME.tar.gz"
+LUA_URL="http://www.lua.org/ftp/$LUA_FILENAME"
+
+if [ $BUILD_JIT ]; then
+  LUA_BASENAME="LuaJIT-2.0.2"
+  LUA_FILENAME="$LUA_BASENAME.tar.gz"
+  LUA_URL="http://luajit.org/download/$LUA_FILENAME"
+fi
+
 # build wxWidgets
 if [ $BUILD_WXWIDGETS ]; then
   svn co "$WXWIDGETS_URL" "$WXWIDGETS_BASENAME" || { echo "Error: failed to checkout wxWidgets"; exit 1; }
@@ -144,10 +166,19 @@ if [ $BUILD_LUA ]; then
   wget -c "$LUA_URL" -O "$LUA_FILENAME" || { echo "Error: failed to download Lua"; exit 1; }
   tar -xzf "$LUA_FILENAME"
   cd "$LUA_BASENAME"
-  make mingw || { echo "Error: failed to build Lua"; exit 1; }
-  make install INSTALL_TOP="$INSTALL_DIR"
-  cp src/lua51.dll "$INSTALL_DIR/lib"
-  [ -f "$INSTALL_DIR/lib/lua51.dll" ] || { echo "Error: lua51.dll isn't found"; exit 1; }
+  if [ $BUILD_JIT ]; then
+    make CCOPT="-DLUAJIT_ENABLE_LUA52COMPAT" || { echo "Error: failed to build Lua"; exit 1; }
+    make install PREFIX="$INSTALL_DIR"
+    cp "$INSTALL_DIR/bin/luajit.exe" "$INSTALL_DIR/bin/lua.exe"
+    # move luajit to lua as it's expected by luasocket and other components
+    cp "$INSTALL_DIR"/include/luajit*/* "$INSTALL_DIR/include/"
+  else
+    make mingw || { echo "Error: failed to build Lua"; exit 1; }
+    make install INSTALL_TOP="$INSTALL_DIR"
+  fi
+  cp src/lua$LUAV.dll "$INSTALL_DIR/lib"
+  cp "$INSTALL_DIR/bin/lua.exe" "$INSTALL_DIR/bin/lua$LUAV.exe"
+  [ -f "$INSTALL_DIR/lib/lua$LUAV.dll" ] || { echo "Error: lua$LUAV.dll isn't found"; exit 1; }
   cd ..
   rm -rf "$LUA_FILENAME" "$LUA_BASENAME"
 fi
@@ -170,9 +201,11 @@ if [ $BUILD_WXLUA ]; then
   # (temporary) fix for compilation issue in wxlua in Windows using mingw (r184)
   sed -i 's/defined(__MINGW32__) || defined(__GNUWIN32__)/0/' modules/wxbind/src/wxcore_bind.cpp
 
-  cp "$INSTALL_DIR/lib/libwxscintilla-2.9.a" "$INSTALL_DIR/lib/libwx_mswu_scintilla-2.9.a"
+  [ -f "$INSTALL_DIR/lib/libwxscintilla-2.9.a" ] && cp "$INSTALL_DIR/lib/libwxscintilla-2.9.a" "$INSTALL_DIR/lib/libwx_mswu_scintilla-2.9.a"
+  [ -f "$INSTALL_DIR/lib/libwxscintilla-3.0.a" ] && cp "$INSTALL_DIR/lib/libwxscintilla-3.0.a" "$INSTALL_DIR/lib/libwx_mswu_scintilla-3.0.a"
+
   echo "set_target_properties(wxLuaModule PROPERTIES LINK_FLAGS -static)" >> modules/luamodule/CMakeLists.txt
-  cmake -G "MSYS Makefiles" -DBUILD_INSTALL_PREFIX="$INSTALL_DIR" -DCMAKE_BUILD_TYPE=$WXLUABUILD -DBUILD_SHARED_LIBS=FALSE \
+  cmake -G "MSYS Makefiles" -DCMAKE_INSTALL_PREFIX="$INSTALL_DIR" -DCMAKE_BUILD_TYPE=$WXLUABUILD -DBUILD_SHARED_LIBS=FALSE \
     -DwxWidgets_CONFIG_EXECUTABLE="$INSTALL_DIR/bin/wx-config" \
     -DwxWidgets_COMPONENTS="stc;html;aui;adv;core;net;base" \
     -DwxLuaBind_COMPONENTS="stc;html;aui;adv;core;net;base" -DwxLua_LUA_LIBRARY_USE_BUILTIN=FALSE \
@@ -189,17 +222,17 @@ if [ $BUILD_LUASOCKET ]; then
   wget --no-check-certificate -c "$LUASOCKET_URL" -O "$LUASOCKET_FILENAME" || { echo "Error: failed to download LuaSocket"; exit 1; }
   unzip "$LUASOCKET_FILENAME"
   cd "$LUASOCKET_BASENAME"
-  mkdir -p "$INSTALL_DIR/lib/lua/5.1/"{mime,socket}
-  gcc $BUILD_FLAGS -o "$INSTALL_DIR/lib/lua/5.1/mime/core.dll" src/mime.c -llua51 \
+  mkdir -p "$INSTALL_DIR/lib/lua/$LUAV/"{mime,socket}
+  gcc $BUILD_FLAGS -o "$INSTALL_DIR/lib/lua/$LUAV/mime/core.dll" src/mime.c -llua$LUAV \
     || { echo "Error: failed to build LuaSocket"; exit 1; }
-  gcc $BUILD_FLAGS -o "$INSTALL_DIR/lib/lua/5.1/socket/core.dll" \
-    src/{auxiliar.c,buffer.c,except.c,inet.c,io.c,luasocket.c,options.c,select.c,tcp.c,timeout.c,udp.c,wsocket.c} -lwsock32 -llua51 \
+  gcc $BUILD_FLAGS -DLUASOCKET_INET_PTON -D_WIN32_WINNT=0x0501 -o "$INSTALL_DIR/lib/lua/$LUAV/socket/core.dll" \
+    src/{auxiliar.c,buffer.c,except.c,inet.c,io.c,luasocket.c,options.c,select.c,tcp.c,timeout.c,udp.c,wsocket.c} -lwsock32 -lws2_32 -llua$LUAV \
     || { echo "Error: failed to build LuaSocket"; exit 1; }
-  mkdir -p "$INSTALL_DIR/share/lua/5.1/socket"
-  cp src/{ftp.lua,http.lua,smtp.lua,tp.lua,url.lua} "$INSTALL_DIR/share/lua/5.1/socket"
-  cp src/{ltn12.lua,mime.lua,socket.lua} "$INSTALL_DIR/share/lua/5.1"
-  [ -f "$INSTALL_DIR/lib/lua/5.1/mime/core.dll" ] || { echo "Error: mime/core.dll isn't found"; exit 1; }
-  [ -f "$INSTALL_DIR/lib/lua/5.1/socket/core.dll" ] || { echo "Error: socket/core.dll isn't found"; exit 1; }
+  mkdir -p "$INSTALL_DIR/share/lua/$LUAV/socket"
+  cp src/{ftp.lua,http.lua,smtp.lua,tp.lua,url.lua} "$INSTALL_DIR/share/lua/$LUAV/socket"
+  cp src/{ltn12.lua,mime.lua,socket.lua} "$INSTALL_DIR/share/lua/$LUAV"
+  [ -f "$INSTALL_DIR/lib/lua/$LUAV/mime/core.dll" ] || { echo "Error: mime/core.dll isn't found"; exit 1; }
+  [ -f "$INSTALL_DIR/lib/lua/$LUAV/socket/core.dll" ] || { echo "Error: socket/core.dll isn't found"; exit 1; }
   cd ..
   rm -rf "$LUASOCKET_FILENAME" "$LUASOCKET_BASENAME"
 fi
@@ -252,20 +285,24 @@ fi
 
 # now copy the compiled dependencies to ZBS binary directory
 mkdir -p "$BIN_DIR" || { echo "Error: cannot create directory $BIN_DIR"; exit 1; }
-[ $BUILD_LUA ] && cp "$INSTALL_DIR/bin/lua.exe" "$INSTALL_DIR/lib/lua51.dll" "$BIN_DIR"
+
+[ $BUILD_LUA ] && cp "$INSTALL_DIR/bin/lua$LUAS.exe" "$INSTALL_DIR/lib/lua$LUAV.dll" "$BIN_DIR"
 [ $BUILD_WXLUA ] && cp "$INSTALL_DIR/bin/libwx.dll" "$BIN_DIR/wx.dll"
 [ $BUILD_WINAPI ] && cp "$INSTALL_DIR/lib/lua/5.1/winapi.dll" "$BIN_DIR"
 [ $BUILD_LUASEC ] && cp "$INSTALL_DIR/lib/lua/5.1/ssl.dll" "$BIN_DIR"
+
 if [ $BUILD_LUASOCKET ]; then
-  mkdir -p "$BIN_DIR/clibs/"{mime,socket}
-  cp "$INSTALL_DIR/lib/lua/5.1/mime/core.dll" "$BIN_DIR/clibs/mime"
-  cp "$INSTALL_DIR/lib/lua/5.1/socket/core.dll" "$BIN_DIR/clibs/socket"
+  mkdir -p "$BIN_DIR/clibs$LUAS/"{mime,socket}
+  cp "$INSTALL_DIR/lib/lua/$LUAV/mime/core.dll" "$BIN_DIR/clibs$LUAS/mime"
+  cp "$INSTALL_DIR/lib/lua/$LUAV/socket/core.dll" "$BIN_DIR/clibs$LUAS/socket"
 fi
 
 # To build lua5.1.dll proxy:
 # (1) get mkforwardlib-gcc.lua from http://lua-users.org/wiki/LuaProxyDllThree
 # (2) run it as "lua mkforwardlib-gcc.lua lua51 lua5.1 X86"
+# To build lua5.2.dll proxy:
+# (1) get mkforwardlib-gcc-52.lua from http://lua-users.org/wiki/LuaProxyDllThree
+# (2) run it as "lua mkforwardlib-gcc-52.lua lua52 lua5.2 X86"
 
-# show a message about successful completion
 echo "*** Build has been successfully completed ***"
 exit 0

cfg/i18n/cn.lua

@@ -159,7 +159,7 @@ return {
   ["Refused a request to start a new debugging session as there is one in progress already."] = "因为有另一个除错在进行，拒绝开启新的除错对话", -- src\editor\debugger.lua
   ["Regular &expression"] = "正则表达式", -- src\editor\findreplace.lua
   ["Remote console"] = "远程主控台", -- src\editor\shellbox.lua
-  ["Replace &All"] = "更换全部", -- src\editor\findreplace.lua
+  ["Replace A&ll"] = "更换全部", -- src\editor\findreplace.lua
   ["Replace"] = "更换", -- src\editor\findreplace.lua
   ["Replaced an invalid UTF8 character with %s."] = "以%s更换无效的UTF8字元", -- src\editor\commands.lua
   ["Replaced"] = "更换", -- src\editor\findreplace.lua

cfg/i18n/de.lua

@@ -163,7 +163,7 @@ return {
   ["Regular &expression"] = "&Regulärer Ausdruck", -- src\editor\findreplace.lua
   ["Remote console"] = "Fensteuerungs-Konsole", -- src\editor\shellbox.lua
   ["Rename All Instances"] = "Umbenennen aller Instanzen", -- src\editor\editor.lua
-  ["Replace &All"] = "&Alles ersetzen", -- src\editor\findreplace.lua
+  ["Replace A&ll"] = "A&lles ersetzen", -- src\editor\findreplace.lua
   ["Replace"] = "Ersetzen", -- src\editor\findreplace.lua
   ["Replaced an invalid UTF8 character with %s."] = "Unbekanntes UTF8-Symbol ersetzt mit %s.", -- src\editor\commands.lua
   ["Replaced"] = "Ersetzt:", -- src\editor\findreplace.lua

cfg/i18n/fr.lua

@@ -163,7 +163,7 @@ return {
   ["Regular &expression"] = "&Expression régulière", -- src\editor\findreplace.lua
   ["Remote console"] = "Console à distance", -- src\editor\shellbox.lua
   ["Rename All Instances"] = "Renommer toutes les occurrences", -- src\editor\editor.lua
-  ["Replace &All"] = "Remplacer &tout", -- src\editor\findreplace.lua
+  ["Replace A&ll"] = "Remplacer &tout", -- src\editor\findreplace.lua
   ["Replace"] = "Remplacer par ", -- src\editor\findreplace.lua
   ["Replaced an invalid UTF8 character with %s."] = "Un caractère UTF8 invalide a été remplacé par %s.", -- src\editor\commands.lua
   ["Replaced"] = "Occurrences remplacées :", -- src\editor\findreplace.lua

cfg/i18n/ru.lua

@@ -160,7 +160,7 @@ return {
   ["Refused a request to start a new debugging session as there is one in progress already."] = "Отказано в запросе на запуск новой отладочной сессии, поскольку одна сессия уже выполняется.", -- src\editor\debugger.lua
   ["Regular &expression"] = "Регулярное выражение", -- src\editor\findreplace.lua
   ["Remote console"] = "Удаленная консоль", -- src\editor\shellbox.lua
-  ["Replace &All"] = "Заменить все", -- src\editor\findreplace.lua
+  ["Replace A&ll"] = "Заменить все", -- src\editor\findreplace.lua
   ["Replace"] = "Заменить", -- src\editor\findreplace.lua
   ["Replaced an invalid UTF8 character with %s."] = "Некорректный символ UTF8 заменен на %s.", -- src\editor\commands.lua
   ["Replaced"] = "Заменено", -- src\editor\findreplace.lua

cfg/scheme-picker.lua

@@ -12,6 +12,7 @@
   - [SolarizedDark](macro:shell(ApplyStyleConfig('cfg/tomorrow.lua','SolarizedDark')))
   - [SolarizedLight](macro:shell(ApplyStyleConfig('cfg/tomorrow.lua','SolarizedLight')))
   - [Notepad++](macro:shell(ApplyStyleConfig('cfg/tomorrow.lua','NotepadPlusPlus')))
+  - [SciTeLuaIDE](macro:shell(ApplyStyleConfig('cfg/tomorrow.lua','SciTeLuaIDE')))
 
   - [ZeroBrane Studio](macro:shell(ide.config.styles = StylesGetDefault(); ReApplySpecAndStyles()))
 

cfg/tomorrow.lua

@@ -4,6 +4,7 @@
 -- Tomorrow colors from https://github.com/chriskempson/tomorrow-theme
 -- Zenburn colors from https://github.com/jnurmine/Zenburn/blob/master/colors/zenburn.vim (contributed by Srdjan Marković)
 -- Monokai colors from http://www.monokai.nl/blog/2006/07/15/textmate-color-theme/
+-- Molokay colors based on https://github.com/tomasr/molokai/blob/master/colors/molokai.vim
 -- Solarized colors from https://github.com/altercation/vim-colors-solarized
 
 local theme = ...
@@ -135,7 +136,7 @@ local colors = {
     Blue        = H'F92672',
     Purple      = H'A6E22E',
   },
-  Molokai = { -- based on https://github.com/tomasr/molokai/blob/master/colors/molokai.vim
+  Molokai = {
     Background  = H'1B1D1E',
     CurrentLine = H'293739',
     Selection   = H'49483E',
@@ -191,6 +192,20 @@ local colors = {
     Blue        = H'2123FF',
     Purple      = H'FFFFFF',
   },
+  SciTeLuaIDE = { -- contributed by Jayanth Acharya
+    Background  = H'1B1D1E',
+    CurrentLine = H'293739',
+    Selection   = H'49483E',
+    Foreground  = H'F8F8F2',    -- super-light-gray (everything else)
+    Comment     = H'00FF00',    -- bright green (comments)
+    Red         = H'F92672',    -- purple (numbers)
+    Orange      = H'AE81FF',    -- lavendar?? (numbers)
+    Yellow      = H'F8F8F2',    -- light-gray
+    Green       = H'FF8000',    -- amber (string literal)
+    Aqua        = H'808080',    -- gray (operators, separators etc.)
+    Blue        = H'0066FF',    -- semi-blue (keywords)
+    Purple      = H'A6E22E',    -- light-grass-green
+  },
 }
 
 -- add more of the specified color (keeping all in 0-255 range)
@@ -251,6 +266,7 @@ return {
   caretlinebg = {bg = C.CurrentLine},
   fold = {fg = C.Comment, bg = C.Background, sel = mixer(C.Comment, 1, 96)},
   whitespace = {fg = C.Comment, bg = C.Background},
+  edge = {},
 
   indicator = {
     fncall = {fg = C.Purple, st = wxstc.wxSTC_INDIC_ROUNDBOX},

cfg/user-sample.lua

@@ -89,9 +89,9 @@ stylesoutshell = styles
 
 -- to change markers used in console and output windows
 styles.marker = styles.marker or {}
-styles.marker.message = {ch = wxstc.wxSTC_MARK_ARROWS, {0, 0, 0}, {240, 240, 240}}
-styles.marker.output = {ch = wxstc.wxSTC_MARK_BACKGROUND, {0, 0, 0}, {240, 240, 240}}
-styles.marker.prompt = {ch = wxstc.wxSTC_MARK_CHARACTER+('>'):byte(), {0, 0, 0}, {240, 240, 240}}
+styles.marker.message = {ch = wxstc.wxSTC_MARK_ARROWS, fg = {0, 0, 0}, bg = {240, 240, 240}}
+styles.marker.output = {ch = wxstc.wxSTC_MARK_BACKGROUND, fg = {0, 0, 0}, bg = {240, 240, 240}}
+styles.marker.prompt = {ch = wxstc.wxSTC_MARK_CHARACTER+('>'):byte(), fg = {0, 0, 0}, bg = {240, 240, 240}}
 stylesoutshell = styles
 
 -- to disable indicators (underlining) on function calls
@@ -103,13 +103,13 @@ styles.indicator.fncall.fg = {240,0,0}
 -- to change the type of the indicator used for function calls
 styles.indicator.fncall.st = wxstc.wxSTC_INDIC_PLAIN
   --[[ other possible values are:
-    wxSTC_INDIC_PLAIN	 Single-line underline
-    wxSTC_INDIC_SQUIGGLE Squiggly underline
-    wxSTC_INDIC_TT	 Line of small T-shapes
-    wxSTC_INDIC_DIAGONAL Diagonal hatching
-    wxSTC_INDIC_STRIKE	 Strike-out
-    wxSTC_INDIC_BOX      Box
-    wxSTC_INDIC_ROUNDBOX Rounded Box
+  wxSTC_INDIC_DOTS   Dotted underline; wxSTC_INDIC_PLAIN       Single-line underline
+  wxSTC_INDIC_TT     Line of Tshapes;  wxSTC_INDIC_SQUIGGLE    Squiggly underline
+  wxSTC_INDIC_STRIKE Strike-out;       wxSTC_INDIC_SQUIGGLELOW Squiggly underline (2 pixels)
+  wxSTC_INDIC_BOX    Box;              wxSTC_INDIC_ROUNDBOX    Rounded Box
+  wxSTC_INDIC_DASH   Dashed underline; wxSTC_INDIC_STRAIGHTBOX Box with trasparency
+  wxSTC_INDIC_DOTBOX Dotted rectangle; wxSTC_INDIC_DIAGONAL    Diagonal hatching
+  wxSTC_INDIC_HIDDEN No visual effect;
   --]]
 
 -- to enable additional spec files (like spec/cpp.lua)

cfg/user-snippet.lua

@@ -1,18 +1,5 @@
 --[[-- Copy snippets from this file to `user.lua` --]]--
 
---[[ Add a shortcut to generate `~` if your keyboard doesn't have one
-local G = ... -- this now points to the global environment in the script
-local ide, wx, TR, ID = G.ide, G.wx, G.TR, G.ID
-local postinit = ide.app.postinit
-ide.app.postinit = function()
-  if postinit then postinit() end
-  local menu = ide.frame.menuBar:GetMenu(ide.frame.menuBar:FindMenu(TR("&Edit")))
-  menu:Append(ID "tilde", "Tilde\tAlt-'")
-  ide.frame:Connect(ID "tilde", wx.wxEVT_COMMAND_MENU_SELECTED,
-    function () GetEditor():AddText("~") end)
-end
---]]
-
 --[[ Add `Evaluate in Console` option to the Edit menu
 local G = ... -- this now points to the global environment in the script
 local ide, wx, TR, ID = G.ide, G.wx, G.TR, G.ID
@@ -28,36 +15,6 @@ ide.app.postinit = function()
 end
 --]]
 
---[[ Add `Zoom` menu to increase/decrease/reset font in the editor
-local G = ... -- this now points to the global environment in the script
-local ide, wx, TR, ID = G.ide, G.wx, G.TR, G.ID
-local postinit = ide.app.postinit
-ide.app.postinit = function()
-  if postinit then postinit() end
-
-  local zoomMenu = wx.wxMenu{
-    {ID "zoomreset", "Zoom to 100%\tCtrl-0"},
-    {ID "zoomin", "Zoom In\tCtrl-+"},
-    {ID "zoomout", "Zoom Out\tCtrl--"},
-  }
-  local menu = ide.frame.menuBar:GetMenu(ide.frame.menuBar:FindMenu(TR("&View")))
-  menu:Append(ID "zoom", "Zoom", zoomMenu)
-
-  ide.frame:Connect(ID "zoomreset", wx.wxEVT_COMMAND_MENU_SELECTED,
-    function () GetEditor():SetZoom(1) end)
-  ide.frame:Connect(ID "zoomin", wx.wxEVT_COMMAND_MENU_SELECTED,
-    function () GetEditor():SetZoom(GetEditor():GetZoom()+1) end)
-  ide.frame:Connect(ID "zoomout", wx.wxEVT_COMMAND_MENU_SELECTED,
-    function () GetEditor():SetZoom(GetEditor():GetZoom()-1) end)
-
-  -- only enable if there is an editor
-  for _, m in G.ipairs({"zoomreset", "zoomin", "zoomout"}) do
-    ide.frame:Connect(ID(m), wx.wxEVT_UPDATE_UI,
-      function (event) event:Enable(GetEditor() ~= nil) end)
-  end
-end
---]]
-
 --[[ An example of how individual keywords can be styled
 local G = ... -- this now points to the global environment in the script
 local luaspec = G.ide.specs['lua']

interpreters/busted.lua

@@ -0,0 +1,66 @@
+-- Copyright 2011-13 Paul Kulchenko, ZeroBrane LLC
+
+local busted
+local win = ide.osname == "Windows"
+
+return {
+  name = "Busted",
+  description = "Busted Lua testing",
+  api = {"baselib"},
+  frun = function(self,wfilename,rundebug)
+    busted = busted or ide.config.path.busted -- check if the path is configured
+    if not busted then
+      local sep = win and ';' or ':'
+      local path = (os.getenv('PATH') or '')..sep
+                 ..(os.getenv('HOME') and os.getenv('HOME') .. '/bin' or '')
+      local paths = {}
+      for p in path:gmatch("[^"..sep.."]+") do
+        busted = busted or GetFullPathIfExists(p, win and 'busted.exe' or 'busted')
+        table.insert(paths, p)
+      end
+      if not busted then
+        DisplayOutput("Can't find busted executable in any of the folders in PATH: "
+          ..table.concat(paths, ", ").."\n")
+        return
+      end
+    end
+
+    local file
+    if rundebug then
+      -- start running the application right away
+      DebuggerAttachDefault({runstart = ide.config.debugger.runonstart == true})
+      local code = (
+[=[xpcall(function() io.stdout:setvbuf('no')
+    require('mobdebug').start(); dofile [[%s]]
+  end, function(err) print(debug.traceback(err)) end)]=])
+        :format(wfilename:GetFullPath())
+      local tmpfile = wx.wxFileName()
+      tmpfile:AssignTempFileName(".")
+      file = tmpfile:GetFullPath()
+      local f = io.open(file, "w")
+      if not f then
+        DisplayOutput("Can't open temporary file '"..file.."' for writing\n")
+        return 
+      end
+      f:write(code)
+      f:close()
+    end
+
+    file = file or wfilename:GetFullPath()
+
+    local options = ide.config.busted and ide.config.busted.options
+      or "--output=TAP"
+    local cmd = ('"%s" %s "%s"'):format(busted, options, file)
+    -- CommandLineRun(cmd,wdir,tooutput,nohide,stringcallback,uid,endcallback)
+    return CommandLineRun(cmd,self:fworkdir(wfilename),true,false,nil,nil,
+      function() ide.debugger.pid = nil if rundebug then wx.wxRemoveFile(file) end end)
+  end,
+  fprojdir = function(self,wfilename)
+    return wfilename:GetPath(wx.wxPATH_GET_VOLUME)
+  end,
+  fworkdir = function(self,wfilename)
+    return ide.config.path.projectdir or wfilename:GetPath(wx.wxPATH_GET_VOLUME)
+  end,
+  hasdebugger = true,
+  fattachdebug = function(self) DebuggerAttachDefault() end,
+}

interpreters/corona.lua

@@ -45,20 +45,34 @@ return {
       DebuggerAttachDefault({startwith = file, redirect = mac and "r" or "c",
         runstart = ide.config.debugger.runonstart ~= false})
 
+      local function needRefresh(mdbl, mdbc)
+        return not wx.wxFileExists(mdbc)
+        or GetFileModTime(mdbc):GetTicks() < GetFileModTime(mdbl):GetTicks()
+      end
+
       -- copy mobdebug.lua to Resources/ folder on Win and to the project folder on OSX
       -- as copying it to Resources/ folder seems to break the signature of the app.
       local mdbc = mac and MergeFullPath(self:fworkdir(wfilename), "mobdebug.lua")
         or MergeFullPath(GetPathWithSep(corona), "Resources/mobdebug.lua")
       local mdbl = MergeFullPath(GetPathWithSep(ide.editorFilename), "lualibs/mobdebug/mobdebug.lua")
-      if not wx.wxFileExists(mdbc)
-      or GetFileModTime(mdbc):GetTicks() < GetFileModTime(mdbl):GetTicks() then
+      local needed = needRefresh(mdbl, mdbc)
+      if needed then
         local copied = FileCopy(mdbl, mdbc)
-        local message = copied
-          and ("Copied debugger ('mobdebug.lua') to '%s'."):format(mdbc)
-          or ("Failed to copy debugger ('mobdebug.lua') to '%s': %s")
-            :format(mdbc, wx.wxSysErrorMsg())
-        DisplayOutputLn(message)
-        if not copied then return end
+        -- couldn't copy to the Resources/ folder; not have permissions?
+        if not copied and win then
+          mdbc = MergeFullPath(wx.wxStandardPaths.Get():GetUserLocalDataDir(),
+            "../../Roaming/Corona Labs/Corona Simulator/Plugins/mobdebug.lua")
+          needed = needRefresh(mdbl, mdbc)
+          copied = needed and FileCopy(mdbl, mdbc)
+        end
+        if needed then
+          local message = copied
+            and ("Copied debugger ('mobdebug.lua') to '%s'."):format(mdbc)
+            or ("Failed to copy debugger ('mobdebug.lua') to '%s': %s")
+              :format(mdbc, wx.wxSysErrorMsg())
+          DisplayOutputLn(message)
+          if not copied then return end
+        end
       end
     end
 

interpreters/gslshell.lua

@@ -55,7 +55,11 @@ return {
       DebuggerAttachDefault({runstart = ide.config.debugger.runonstart == true})
     end
 
-    local cmd = ('"%s" "%s"'):format(gslshell, wfilename:GetFullPath())
+    local code = rundebug
+      and ([[-e "io.stdout:setvbuf('no'); %s"]]):format(rundebug)
+       or ([[-e "io.stdout:setvbuf('no')" "%s"]]):format(wfilename:GetFullPath())
+    local cmd = '"'..gslshell..'" '..code
+
     -- CommandLineRun(cmd,wdir,tooutput,nohide,stringcallback,uid,endcallback)
     return CommandLineRun(cmd,self:fworkdir(wfilename),true,false,nil,nil,
       function() ide.debugger.pid = nil end)
@@ -70,4 +74,5 @@ return {
   fattachdebug = function(self) DebuggerAttachDefault() end,
   skipcompile = true,
   unhideanywindow = true,
+  scratchextloop = false,
 }

interpreters/luabase.lua

@@ -0,0 +1,69 @@
+function MakeLuaInterpreter(version, name)
+
+local exe
+
+local function exePath(self, version)
+  local version = tostring(version):gsub('%.','')
+  local mainpath = ide.editorFilename:gsub("[^/\\]+$","")
+  local macExe = mainpath..([[bin/lua.app/Contents/MacOS/lua%s]]):format(version)
+  return ide.config.path['lua'..version]
+     or (ide.osname == "Windows" and mainpath..([[bin\lua%s.exe]]):format(version))
+     or (ide.osname == "Unix" and mainpath..([[bin/linux/%s/lua%s]]):format(ide.osarch, version))
+     or (wx.wxFileExists(macExe) and macExe or mainpath..([[bin/lua%s]]):format(version))
+end
+
+return {
+  name = ("Lua%s"):format(name or version or ""),
+  description = ("Lua%s interpreter with debugger"):format(name or version or ""),
+  api = {"wxwidgets","baselib"},
+  luaversion = version or '5.1',
+  fexepath = exePath,
+  frun = function(self,wfilename,rundebug)
+    exe = exe or self:fexepath(version or "")
+    local filepath = wfilename:GetFullPath()
+    if rundebug then
+      DebuggerAttachDefault({runstart = ide.config.debugger.runonstart == true})
+    else
+      -- if running on Windows and can't open the file, this may mean that
+      -- the file path includes unicode characters that need special handling
+      local fh = io.open(filepath, "r")
+      if fh then fh:close() end
+      if ide.osname == 'Windows' and pcall(require, "winapi")
+      and wfilename:FileExists() and not fh then
+        winapi.set_encoding(winapi.CP_UTF8)
+        filepath = winapi.short_path(filepath)
+      end
+    end
+    local code = rundebug
+      and ([[-e "io.stdout:setvbuf('no'); %s"]]):format(rundebug)
+       or ([[-e "io.stdout:setvbuf('no')" "%s"]]):format(filepath)
+    local cmd = '"'..exe..'" '..code
+
+    -- modify CPATH to work with other Lua versions
+    local clibs = ('/clibs%s/'):format(version and tostring(version):gsub('%.','') or '')
+    local _, cpath = wx.wxGetEnv("LUA_CPATH")
+    if version and cpath and not cpath:find(clibs, 1, true) then
+      wx.wxSetEnv("LUA_CPATH", cpath:gsub('/clibs/', clibs)) end
+
+    -- CommandLineRun(cmd,wdir,tooutput,nohide,stringcallback,uid,endcallback)
+    local pid = CommandLineRun(cmd,self:fworkdir(wfilename),true,false,nil,nil,
+      function() ide.debugger.pid = nil end)
+
+    if version and cpath then wx.wxSetEnv("LUA_CPATH", cpath) end
+    return pid
+  end,
+  fprojdir = function(self,wfilename)
+    return wfilename:GetPath(wx.wxPATH_GET_VOLUME)
+  end,
+  fworkdir = function (self,wfilename)
+    return ide.config.path.projectdir or wfilename:GetPath(wx.wxPATH_GET_VOLUME)
+  end,
+  hasdebugger = true,
+  fattachdebug = function(self) DebuggerAttachDefault() end,
+  scratchextloop = false,
+  unhideanywindow = true,
+}
+
+end
+
+return nil -- as this is not a real interpreter

interpreters/luadeb.lua

@@ -1,50 +1,2 @@
-local exe
-
-local function exePath()
-  local mainpath = ide.editorFilename:gsub("[^/\\]+$","")
-  local macExe = mainpath..'bin/lua.app/Contents/MacOS/lua'
-  return ide.config.path.lua or
-        (ide.osname == "Windows" and mainpath..[[bin\lua.exe]]
-     or (ide.osname == "Unix" and mainpath..([[bin/linux/%s/lua]]):format(ide.osarch))
-     or (wx.wxFileExists(macExe) and macExe or mainpath..[[bin/lua]]))
-end
-
-return {
-  name = "Lua",
-  description = "Lua interpreter with debugger",
-  api = {"wxwidgets","baselib"},
-  frun = function(self,wfilename,rundebug)
-    exe = exe or exePath()
-    local filepath = wfilename:GetFullPath()
-    if rundebug then
-      DebuggerAttachDefault({runstart = ide.config.debugger.runonstart == true})
-    else
-      -- if running on Windows and can't open the file, this may mean that
-      -- the file path includes unicode characters that need special handling
-      local fh = io.open(filepath, "r")
-      if fh then fh:close() end
-      if ide.osname == 'Windows' and pcall(require, "winapi")
-      and wfilename:FileExists() and not fh then
-        winapi.set_encoding(winapi.CP_UTF8)
-        filepath = winapi.short_path(filepath)
-      end
-    end
-    local code = rundebug
-      and ([[-e "io.stdout:setvbuf('no'); %s"]]):format(rundebug)
-       or ([[-e "io.stdout:setvbuf('no')" "%s"]]):format(filepath)
-    local cmd = '"'..exe..'" '..code
-    -- CommandLineRun(cmd,wdir,tooutput,nohide,stringcallback,uid,endcallback)
-    return CommandLineRun(cmd,self:fworkdir(wfilename),true,false,nil,nil,
-      function() ide.debugger.pid = nil end)
-  end,
-  fprojdir = function(self,wfilename)
-    return wfilename:GetPath(wx.wxPATH_GET_VOLUME)
-  end,
-  fworkdir = function (self,wfilename)
-    return ide.config.path.projectdir or wfilename:GetPath(wx.wxPATH_GET_VOLUME)
-  end,
-  hasdebugger = true,
-  fattachdebug = function(self) DebuggerAttachDefault() end,
-  scratchextloop = false,
-  unhideanywindow = true,
-}
+dofile 'interpreters/luabase.lua'
+return MakeLuaInterpreter()

interpreters/luadeb52.lua

@@ -0,0 +1,2 @@
+dofile 'interpreters/luabase.lua'
+return MakeLuaInterpreter(5.2, ' 5.2')

interpreters/marmalade.lua

@@ -18,7 +18,8 @@ return {
            win and ([[C:\Marmalade]]..sep..[[D:\Marmalade]]..sep..
                     [[C:\Program Files\Marmalade]]..sep..[[D:\Program Files\Marmalade]]..sep..
                     [[C:\Program Files (x86)\Marmalade]]..sep..[[D:\Program Files (x86)\Marmalade]]..sep)
-        or mac and ([[/Developer/Marmalade]]..sep)
+        or mac and ([[/Applications/Marmalade.app/Contents]]..sep..
+                    [[/Developer/Marmalade]]..sep)
         or ''
       -- Marmalade can be installed in a folder with version number or without
       -- so it may be c:\Marmalade\s3e\... or c:\Marmalade\6.2\s3e\...
@@ -29,6 +30,8 @@ return {
           if GetFullPathIfExists(candidate, exe) then table.insert(candidates, candidate) end
           if GetFullPathIfExists(candidate.."/s3e", exe) then table.insert(candidates, candidate.."/s3e") end
         end
+        -- stop on Mac if found something in /Applications (7.0+)
+        if mac and #candidates > 0 then break end
       end
       -- multiple candidates may be present, so sort and use the latest.
       -- only happens if multiple versions are installed and S3E_DIR is not set.
@@ -43,6 +46,8 @@ return {
       end
     end
 
+    if not s3e then s3e = quick:gsub(exe, '') end
+
     local projdir = self:fworkdir(wfilename)
     local file = GetFullPathIfExists(projdir, 'main.lua')
     if not file then

lualibs/copas/copas.lua

@@ -19,9 +19,10 @@ end
 
 local socket = require "socket"
 
-require "coxpcall"
+local coxpcall = require "coxpcall"
 
 local WATCH_DOG_TIMEOUT = 120
+local UDP_DATAGRAM_MAX = 8192
 
 -- Redefines LuaSocket functions with coroutine safe versions
 -- (this allows the use of socket.http from within copas)
@@ -37,7 +38,7 @@ end
 
 function socket.protect(func)
   return function (...)
-           return statusHandler(copcall(func, ...))
+           return statusHandler(coxpcall.pcall(func, ...))
          end
 end
 
@@ -45,7 +46,7 @@ function socket.newtry(finalizer)
   return function (...)
            local status = (...)
            if not status then
-             copcall(finalizer, select(2, ...))
+             coxpcall.pcall(finalizer, select(2, ...))
              error({ (select(2, ...)) }, 0)
            end
            return ...
@@ -54,16 +55,15 @@ end
 
 -- end of LuaSocket redefinitions
 
-
-module ("copas", package.seeall)
+local copas = {}
 
 -- Meta information is public even if beginning with an "_"
-_COPYRIGHT   = "Copyright (C) 2005-2010 Kepler Project"
-_DESCRIPTION = "Coroutine Oriented Portable Asynchronous Services"
-_VERSION     = "Copas 1.1.7"
+copas._COPYRIGHT   = "Copyright (C) 2005-2010 Kepler Project"
+copas._DESCRIPTION = "Coroutine Oriented Portable Asynchronous Services"
+copas._VERSION     = "Copas 1.1.7"
 
 -- Close the socket associated with the current connection after the handler finishes
-autoclose = true
+copas.autoclose = true
 
 -------------------------------------------------------------------------------
 -- Simple set implementation based on LuaSocket's tinyirc.lua example
@@ -128,7 +128,10 @@ local _writing = newset() -- sockets currently being written
 -- Coroutine based socket I/O functions.
 -------------------------------------------------------------------------------
 -- reads a pattern from a client and yields to the reading set on timeouts
-function receive(client, pattern, part)
+-- UDP: a UDP socket expects a second argument to be a number, so it MUST
+-- be provided as the 'pattern' below defaults to a string. Will throw a
+-- 'bad argument' error if omitted.
+function copas.receive(client, pattern, part)
   local s, err
   pattern = pattern or "*l"
   repeat
@@ -142,9 +145,25 @@ function receive(client, pattern, part)
   until false
 end
 
+-- receives data from a client over UDP. Not available for TCP.
+-- (this is a copy of receive() method, adapted for receivefrom() use)
+function copas.receivefrom(client, size)
+  local s, err, port
+  size = size or UDP_DATAGRAM_MAX
+  repeat
+    s, err, port = client:receivefrom(size) -- upon success err holds ip address
+    if s or err ~= "timeout" then
+      _reading_log[client] = nil
+      return s, err, port
+    end
+    _reading_log[client] = os.time()
+    coroutine.yield(client, _reading)
+  until false
+end
+
 -- same as above but with special treatment when reading chunks,
 -- unblocks on any data received.
-function receivePartial(client, pattern)
+function copas.receivePartial(client, pattern)
   local s, err, part
   pattern = pattern or "*l"
   repeat
@@ -161,7 +180,8 @@ end
 
 -- sends data to a client. The operation is buffered and
 -- yields to the writing set on timeouts
-function send(client,data, from, to)
+-- Note: from and to parameters will be ignored by/for UDP sockets
+function copas.send(client, data, from, to)
   local s, err,sent
   from = from or 1
   local lastIndex = from - 1
@@ -183,8 +203,30 @@ function send(client,data, from, to)
   until false
 end
 
+-- sends data to a client over UDP. Not available for TCP.
+-- (this is a copy of send() method, adapted for sendto() use)
+function copas.sendto(client, data, ip, port)
+  local s, err,sent
+
+  repeat
+    s, err = client:sendto(data, ip, port)
+    -- adds extra corrotine swap
+    -- garantees that high throuput dont take other threads to starvation
+    if (math.random(100) > 90) then
+      _writing_log[client] = os.time()
+      coroutine.yield(client, _writing)
+    end
+    if s or err ~= "timeout" then
+      _writing_log[client] = nil
+      return s, err
+    end
+    _writing_log[client] = os.time()
+    coroutine.yield(client, _writing)
+  until false
+end
+
 -- waits until connection is completed
-function connect(skt, host, port)
+function copas.connect(skt, host, port)
   skt:settimeout(0)
   local ret, err
   repeat
@@ -200,24 +242,24 @@ function connect(skt, host, port)
 end
 
 -- flushes a client write buffer (deprecated)
-function flush(client)
+function copas.flush(client)
 end
 
--- wraps a socket to use Copas methods (send, receive, flush and settimeout)
+-- wraps a TCP socket to use Copas methods (send, receive, flush and settimeout)
 local _skt_mt = {__index = {
                    send = function (self, data, from, to)
-                            return send (self.socket, data, from, to)
+                            return copas.send (self.socket, data, from, to)
                           end,
 
                    receive = function (self, pattern)
                                if (self.timeout==0) then
-                                 return receivePartial(self.socket, pattern)
+                                 return copas.receivePartial(self.socket, pattern)
                                end
-                               return receive (self.socket, pattern)
+                               return copas.receive(self.socket, pattern)
                              end,
 
                    flush = function (self)
-                             return flush (self.socket)
+                             return copas.flush(self.socket)
                            end,
 
                    settimeout = function (self,time)
@@ -226,8 +268,41 @@ local _skt_mt = {__index = {
                                 end,
                }}
 
-function wrap (skt)
-  return  setmetatable ({socket = skt}, _skt_mt)
+-- wraps a UDP socket, copy of TCP one adapted for UDP.
+-- Mainly adds sendto() and receivefrom()
+local _skt_mt_udp = {__index = {
+                   send = function (self, data)
+                            return copas.send (self.socket, data)
+                          end,
+
+                   sendto = function (self, data, ip, port)
+                            return copas.sendto (self.socket, data, ip, port)
+                          end,
+
+                   receive = function (self, size)
+                               return copas.receive (self.socket, (size or UDP_DATAGRAM_MAX))
+                             end,
+
+                   receivefrom = function (self, size)
+                               return copas.receivefrom (self.socket, (size or UDP_DATAGRAM_MAX))
+                             end,
+
+                   flush = function (self)
+                             return copas.flush (self.socket)
+                           end,
+
+                   settimeout = function (self,time)
+                                  self.timeout=time
+                                  return
+                                end,
+               }}
+
+function copas.wrap (skt)
+  if string.sub(tostring(skt),1,3) == "udp" then
+    return  setmetatable ({socket = skt}, _skt_mt_udp)
+  else
+    return  setmetatable ({socket = skt}, _skt_mt)
+  end
 end
 
 --------------------------------------------------
@@ -236,7 +311,7 @@ end
 
 local _errhandlers = {}   -- error handler per coroutine
 
-function setErrorHandler (err)
+function copas.setErrorHandler (err)
   local co = coroutine.running()
   if co then
     _errhandlers [co] = err
@@ -260,8 +335,8 @@ local function _doTick (co, skt, ...)
     new_q:insert (res)
     new_q:push (res, co)
   else
-    if not ok then copcall (_errhandlers [co] or _deferror, res, co, skt) end
-    if skt and autoclose then skt:close() end
+    if not ok then coxpcall.pcall (_errhandlers [co] or _deferror, res, co, skt) end
+    if skt and copas.autoclose then skt:close() end
     _errhandlers [co] = nil
   end
 end
@@ -303,7 +378,7 @@ local function addUDPserver(server, handler, timeout)
     _doTick (co, server)
 end
 
-function addserver(server, handler, timeout)
+function copas.addserver(server, handler, timeout)
     if string.sub(tostring(server),1,3) == "udp" then
         addUDPserver(server, handler, timeout)
     else
@@ -313,7 +388,7 @@ end
 -------------------------------------------------------------------------------
 -- Adds an new courotine thread to Copas dispatcher
 -------------------------------------------------------------------------------
-function addthread(thread, ...)
+function copas.addthread(thread, ...)
   if type(thread) ~= "thread" then
     thread = coroutine.create(thread)
   end
@@ -397,10 +472,6 @@ local last_cleansing = 0
 -------------------------------------------------------------------------------
 local function _select (timeout)
   local err
-  local readable={}
-  local writable={}
-  local r={}
-  local w={}
   local now = os.time()
   local duration = os.difftime
 
@@ -437,10 +508,12 @@ end
 -------------------------------------------------------------------------------
 -- Dispatcher loop step.
 -- Listen to client requests and handles them
+-- Returns false if no data was handled (timeout), or true if there was data
+-- handled (or nil + error message)
 -------------------------------------------------------------------------------
-function step(timeout)
+function copas.step(timeout)
   local err = _select (timeout)
-  if err == "timeout" then return end
+  if err == "timeout" then return false end
 
   if err then
     error(err)
@@ -451,14 +524,17 @@ function step(timeout)
       tsk:tick (ev)
     end
   end
+  return true
 end
 
 -------------------------------------------------------------------------------
 -- Dispatcher endless loop.
 -- Listen to client requests and handles them forever
 -------------------------------------------------------------------------------
-function loop(timeout)
+function copas.loop(timeout)
   while true do
-    step(timeout)
+    copas.step(timeout)
   end
 end
+
+return copas

lualibs/coxpcall/coxpcall.lua

@@ -5,31 +5,19 @@
 -- be dealed without the usual Lua 5.x pcall/xpcall issues with coroutines
 -- yielding inside the call to pcall or xpcall.
 --
--- Authors: Roberto Ierusalimschy and Andre Carregal 
--- Contributors: Thomas Harning Jr., Ignacio Burgueño, Fábio Mascarenhas
+-- Authors: Roberto Ierusalimschy and Andre Carregal
+-- Contributors: Thomas Harning Jr., Ignacio Burgueño, Fabio Mascarenhas
 --
 -- Copyright 2005 - Kepler Project (www.keplerproject.org)
 --
 -- $Id: coxpcall.lua,v 1.13 2008/05/19 19:20:02 mascarenhas Exp $
 -------------------------------------------------------------------------------
 
--- This version has been modified to handle the case where a child coroutine
--- (created by copcall or coxpcall) is directly resumed, instead of resuming
--- its parent coroutine, because the coroutine reference was obtained by
--- calling coroutine.running from the child coroutine. The modification
--- implemented here is to override coroutine.running so that it returns the
--- parent coroutine instead of the child one.
-
-local oldrunning = coroutine.running
-local parents = {}
-setmetatable( parents, { __mode = "kv" } )
-
-local function getRoot( co )
-	return parents[co] or co
-end
-
-coroutine.running = function()
-	return getRoot( oldrunning() )
+-- Lua 5.2 makes this module a no-op
+if _VERSION == "Lua 5.2" then
+  copcall = pcall
+  coxpcall = xpcall
+  return { pcall = pcall, xpcall = xpcall }
 end
 
 -------------------------------------------------------------------------------
@@ -37,7 +25,9 @@ end
 -------------------------------------------------------------------------------
 local performResume, handleReturnValue
 local oldpcall, oldxpcall = pcall, xpcall
-
+local pack = table.pack or function(...) return {n = select("#", ...), ...} end
+local unpack = table.unpack or unpack
+  
 function handleReturnValue(err, co, status, ...)
     if not status then
         return false, err(debug.traceback(co, (...)), ...)
@@ -51,16 +41,15 @@ end
 
 function performResume(err, co, ...)
     return handleReturnValue(err, co, coroutine.resume(co, ...))
-end    
+end
 
 function coxpcall(f, err, ...)
     local res, co = oldpcall(coroutine.create, f)
     if not res then
-        local params = {...}
-        local newf = function() return f(unpack(params)) end
+        local params = pack(...)
+        local newf = function() return f(unpack(params, 1, params.n)) end
         co = coroutine.create(newf)
     end
-	parents[co] = getRoot( oldrunning() )
     return performResume(err, co, ...)
 end
 
@@ -76,6 +65,4 @@ function copcall(f, ...)
     return coxpcall(f, id, ...)
 end
 
--------------------------------------------------------------------------------
--- Override coroutine.running
--------------------------------------------------------------------------------
+return { pcall = copcall, xpcall = coxpcall }

lualibs/dist/config.lua

@@ -0,0 +1,112 @@
+-- Luadist configuration
+
+module ("dist.config", package.seeall)
+
+local sys = require "dist.sys"
+local utils = require "dist.utils"
+local win = (os.getenv('WINDIR') or (os.getenv('OS') or ''):match('[Ww]indows'))
+  and not (os.getenv('OSTYPE') or ''):match('cygwin') -- exclude cygwin
+
+-- System information ------------------------------------------------
+version       = "0.2.7"   -- Current LuaDist version
+-- set initial architecture as it's important for path separators
+arch          = win and "Windows" or "Linux" -- Host architecture
+type          = "x86"      -- Host type
+
+-- Directories -------------------------------------------------------
+root_dir      = os.getenv("DIST_ROOT") or utils.get_luadist_location() or sys.path_separator()
+temp_dir      = "tmp"
+cache_dir     = sys.make_path(temp_dir, "cache")
+distinfos_dir = sys.make_path("share", "luadist-git", "dists")
+test_dir      = sys.make_path("share", "luadist-git", "test")
+
+-- Files -------------------------------------------------------------
+manifest_file  = sys.make_path(cache_dir, ".gitmodules")
+dep_cache_file = sys.make_path(cache_dir, ".depcache")
+log_file       = sys.make_path(temp_dir, "luadist.log")
+cache_file     = ""
+
+-- Repositories ------------------------------------------------------
+repos = {
+    "git://github.com/LuaDist/Repository.git",
+}
+
+upload_url = "git@github.com:LuaDist" -- must not contain trailing '/'
+
+-- Settings ----------------------------------------------------------
+debug         = false         -- Use debug mode.
+verbose       = false         -- Print verbose output.
+simulate      = false         -- Only simulate installation of packages.
+binary        = true          -- Use binary version of modules.
+source        = true          -- Use source version of modules.
+test          = false         -- Run CTest before install.
+
+cache         = true          -- Use cache.
+cache_timeout = 3 * 60 * 60   -- Cache timeout in seconds.
+
+dep_cache     = true          -- Use cache for dependency information (tree functionality).
+
+-- Components (of modules) that will be installed.
+components    = {
+  "Runtime", "Library", "Header", "Data", "Documentation", "Example", "Test", "Other", "Unspecified"
+}
+
+-- Available log levels are: DEBUG, INFO, WARN, ERROR, FATAL (see dist.logger for more information).
+print_log_level = "WARN"      -- Minimum level for log messages to be printed (nil to disable).
+write_log_level = "INFO"      -- Minimum level for log messages to be logged (nil to disable).
+
+
+-- CMake variables ---------------------------------------------------
+variables = {
+  --- Install defaults
+  INSTALL_BIN                        = "bin",
+  INSTALL_LIB                        = "lib",
+  INSTALL_INC                        = "include",
+  INSTALL_ETC                        = "etc",
+  INSTALL_LMOD                       = "lib/lua",
+  INSTALL_CMOD                       = "lib/lua",
+
+  --- LuaDist specific variables
+  DIST_VERSION                       = version,
+  DIST_ARCH                          = arch,
+  DIST_TYPE                          = type,
+
+  -- CMake specific setup
+  CMAKE_GENERATOR                    = win and "MinGW Makefiles" or "Unix Makefiles",
+  CMAKE_BUILD_TYPE                   = "MinSizeRel",
+
+  -- RPath functionality
+  CMAKE_SKIP_BUILD_RPATH             = "FALSE",
+  CMAKE_BUILD_WITH_INSTALL_RPATH     = "FALSE",
+  CMAKE_INSTALL_RPATH                = "$ORIGIN/../lib",
+  CMAKE_INSTALL_RPATH_USE_LINK_PATH  = "TRUE",
+  CMAKE_INSTALL_NAME_DIR             = "@executable_path/../lib",
+
+  -- OSX specific
+  CMAKE_OSX_ARCHITECTURES            = "",
+}
+
+-- Building ----------------------------------------------------------
+cmake         = "cmake"
+ctest         = "ctest"
+
+cache_command = cmake .. " -C cache.cmake"
+build_command = cmake .. " --build . --clean-first"
+
+install_component_command = " -DCOMPONENT=#COMPONENT# -P cmake_install.cmake"
+
+test_command = ctest .. " -V ."
+
+strip_option = " -DCMAKE_INSTALL_DO_STRIP=true"
+cache_debug_options = "-DCMAKE_VERBOSE_MAKEFILE=true -DCMAKE_BUILD_TYPE=Debug"
+build_debug_options = ""
+
+-- Add -j option to make in case of unix makefiles to speed up builds
+if (variables.CMAKE_GENERATOR == "Unix Makefiles") then
+        build_command = build_command .. " -- -j6"
+end
+
+-- Add -j option to make in case of MinGW makefiles to speed up builds
+if (variables.CMAKE_GENERATOR == "MinGW Makefiles") then
+        build_command = "set SHELL=cmd.exe && " .. build_command .. " -- -j"
+end

lualibs/dist/constraints.lua

@@ -0,0 +1,271 @@
+-- Note: the code of this module is borrowed from the original LuaDist project
+
+
+
+--- LuaDist version constraints functions
+-- Peter Drahoš, LuaDist Project, 2010
+-- Original Code borrowed from LuaRocks Project
+
+--- Version constraints handling functions.
+-- Dependencies are represented in LuaDist through strings with
+-- a dist name followed by a comma-separated list of constraints.
+-- Each constraint consists of an operator and a version number.
+-- In this string format, version numbers are represented as
+-- naturally as possible, like they are used by upstream projects
+-- (e.g. "2.0beta3"). Internally, LuaDist converts them to a purely
+-- numeric representation, allowing comparison following some
+-- "common sense" heuristics. The precise specification of the
+-- comparison criteria is the source code of this module, but the
+-- test/test_deps.lua file included with LuaDist provides some
+-- insights on what these criteria are.
+
+module ("dist.constraints", package.seeall)
+
+
+local operators = {
+  ["=="]  = "==",
+  ["~="]  = "~=",
+  [">"]   = ">",
+  ["<"]   = "<",
+  [">="]  = ">=",
+  ["<="]  = "<=",
+  ["~>"]  = "~>",
+  -- plus some convenience translations
+  [""]  = "==",
+  ["-"] = "==",
+  ["="]   = "==",
+  ["!="]  = "~="
+}
+
+local deltas = {
+   scm =   -100,
+   rc =    -1000,
+   pre =   -10000,
+   beta =  -100000,
+   alpha = -1000000,
+   work =  -10000000,
+}
+
+local version_mt = {
+   --- Equality comparison for versions.
+   -- All version numbers must be equal.
+   -- If both versions have revision numbers, they must be equal;
+   -- otherwise the revision number is ignored.
+   -- @param v1 table: version table to compare.
+   -- @param v2 table: version table to compare.
+   -- @return boolean: true if they are considered equivalent.
+   __eq = function(v1, v2)
+      if #v1 ~= #v2 then
+         return false
+      end
+      for i = 1, #v1 do
+         if v1[i] ~= v2[i] then
+            return false
+         end
+      end
+      if v1.revision and v2.revision then
+         return (v1.revision == v2.revision)
+      end
+      return true
+   end,
+   --- Size comparison for versions.
+   -- All version numbers are compared.
+   -- If both versions have revision numbers, they are compared;
+   -- otherwise the revision number is ignored.
+   -- @param v1 table: version table to compare.
+   -- @param v2 table: version table to compare.
+   -- @return boolean: true if v1 is considered lower than v2.
+   __lt = function(v1, v2)
+      for i = 1, math.max(#v1, #v2) do
+         local v1i, v2i = v1[i] or 0, v2[i] or 0
+         if v1i ~= v2i then
+            return (v1i < v2i)
+         end
+      end
+      if v1.revision and v2.revision then
+         return (v1.revision < v2.revision)
+      end
+      return false
+   end
+}
+
+local version_cache = {}
+setmetatable(version_cache, {
+   __mode = "kv"
+})
+
+--- Parse a version string, converting to table format.
+-- A version table contains all components of the version string
+-- converted to numeric format, stored in the array part of the table.
+-- If the version contains a revision, it is stored numerically
+-- in the 'revision' field. The original string representation of
+-- the string is preserved in the 'string' field.
+-- Returned version tables use a metatable
+-- allowing later comparison through relational operators.
+-- @param vstring string: A version number in string format.
+-- @return table or nil: A version table or nil
+-- if the input string contains invalid characters.
+function parseVersion(vstring)
+  if not vstring then return nil end
+  assert(type(vstring) == "string")
+
+  local cached = version_cache[vstring]
+  if cached then
+    return cached
+  end
+
+  local version = {}
+  local i = 1
+
+  local function add_token(number)
+    version[i] = version[i] and version[i] + number/100000 or number
+    i = i + 1
+  end
+
+  -- trim leading and trailing spaces
+  vstring = vstring:match("^%s*(.*)%s*$")
+  version.string = vstring
+  -- store revision separately if any
+  local main, revision = vstring:match("(.*)%-(%d+)$")
+  if revision then
+    vstring = main
+    version.revision = tonumber(revision)
+  end
+  while #vstring > 0 do
+    -- extract a number
+    local token, rest = vstring:match("^(%d+)[%.%-%_]*(.*)")
+    if token then
+      add_token(tonumber(token))
+    else
+      -- extract a word
+      token, rest = vstring:match("^(%a+)[%.%-%_]*(.*)")
+      if not token then
+        return nil
+      end
+      local last = #version
+      version[i] = deltas[token] or (token:byte() / 1000)
+    end
+    vstring = rest
+  end
+  setmetatable(version, version_mt)
+  version_cache[vstring] = version
+  return version
+end
+
+--- Utility function to compare version numbers given as strings.
+-- @param a string: one version.
+-- @param b string: another version.
+-- @return boolean: True if a > b.
+function compareVersions(a, b)
+  return parseVersion(a) > parseVersion(b)
+end
+
+--- Consumes a constraint from a string, converting it to table format.
+-- For example, a string ">= 1.0, > 2.0" is converted to a table in the
+-- format {op = ">=", version={1,0}} and the rest, "> 2.0", is returned
+-- back to the caller.
+-- @param input string: A list of constraints in string format.
+-- @return (table, string) or nil: A table representing the same
+-- constraints and the string with the unused input, or nil if the
+-- input string is invalid.
+local function parseConstraint(input)
+  assert(type(input) == "string")
+
+  local op, version, rest = input:match("^([<>=~!]*)%s*([%w%.%_%-]+)[%s,]*(.*)")
+  op = operators[op]
+  version = parseVersion(version)
+  if not op or not version then return nil end
+  return { op = op, version = version }, rest
+end
+
+--- Convert a list of constraints from string to table format.
+-- For example, a string ">= 1.0, < 2.0" is converted to a table in the format
+-- {{op = ">=", version={1,0}}, {op = "<", version={2,0}}}.
+-- Version tables use a metatable allowing later comparison through
+-- relational operators.
+-- @param input string: A list of constraints in string format.
+-- @return table or nil: A table representing the same constraints,
+-- or nil if the input string is invalid.
+function parseConstraints(input)
+  assert(type(input) == "string")
+
+  local constraints, constraint = {}, nil
+  while #input > 0 do
+    constraint, input = parseConstraint(input)
+    if constraint then
+      table.insert(constraints, constraint)
+      else
+      return nil
+    end
+  end
+  return constraints
+end
+
+--- A more lenient check for equivalence between versions.
+-- This returns true if the requested components of a version
+-- match and ignore the ones that were not given. For example,
+-- when requesting "2", then "2", "2.1", "2.3.5-9"... all match.
+-- When requesting "2.1", then "2.1", "2.1.3" match, but "2.2"
+-- doesn't.
+-- @param version string or table: Version to be tested; may be
+-- in string format or already parsed into a table.
+-- @param requested string or table: Version requested; may be
+-- in string format or already parsed into a table.
+-- @return boolean: True if the tested version matches the requested
+-- version, false otherwise.
+local function partialMatch(version, requested)
+  assert(type(version) == "string" or type(version) == "table")
+  assert(type(requested) == "string" or type(version) == "table")
+
+  if type(version) ~= "table" then version = parseVersion(version) end
+  if type(requested) ~= "table" then requested = parseVersion(requested) end
+  if not version or not requested then return false end
+
+  for i = 1, #requested do
+    if requested[i] ~= version[i] then return false end
+  end
+  if requested.revision then
+    return requested.revision == version.revision
+  end
+  return true
+end
+
+--- Check if a version satisfies a set of constraints.
+-- @param version table: A version in table format
+-- @param constraints table: An array of constraints in table format.
+-- @return boolean: True if version satisfies all constraints,
+-- false otherwise.
+function matchConstraints(version, constraints)
+  assert(type(version) == "table")
+  assert(type(constraints) == "table")
+  local ok = true
+  setmetatable(version, version_mt)
+  for _, constr in pairs(constraints) do
+    local constr_version = constr.version
+          setmetatable(constr.version, version_mt)
+    if     constr.op == "==" then ok = version == constr_version
+    elseif constr.op == "~=" then ok = version ~= constr_version
+    elseif constr.op == ">"  then ok = version >  constr_version
+    elseif constr.op == "<"  then ok = version <  constr_version
+    elseif constr.op == ">=" then ok = version >= constr_version
+    elseif constr.op == "<=" then ok = version <= constr_version
+    elseif constr.op == "~>" then ok = partialMatch(version, constr_version)
+    end
+    if not ok then break end
+  end
+  return ok
+end
+
+--- Check if a version string is satisfied by a constraint string.
+-- @param version string: A version in string format
+-- @param constraints string: Constraints in string format.
+-- @return boolean: True if version satisfies all constraints,
+-- false otherwise.
+function constraint_satisfied(version, constraints)
+  local const = parseConstraints(constraints)
+  local ver = parseVersion(version)
+  if const and ver then
+    return matchConstraints(ver, const)
+  end
+  return nil, "Error parsing versions."
+end

lualibs/dist/depends.lua

@@ -0,0 +1,770 @@
+-- Utility functions for dependencies
+
+module ("dist.depends", package.seeall)
+
+local cfg = require "dist.config"
+local mf = require "dist.manifest"
+local sys = require "dist.sys"
+local const = require "dist.constraints"
+local utils = require "dist.utils"
+local package = require "dist.package"
+
+-- Return all packages with specified names from manifest.
+-- Names can also contain version constraint (e.g. 'copas>=1.2.3', 'saci-1.0' etc.).
+function find_packages(package_names, manifest)
+    if type(package_names) == "string" then package_names = {package_names} end
+    manifest = manifest or mf.get_manifest()
+    assert(type(package_names) == "table", "depends.find_packages: Argument 'package_names' is not a table or string.")
+    assert(type(manifest) == "table", "depends.find_packages: Argument 'manifest' is not a table.")
+
+    local packages_found = {}
+    -- find matching packages in manifest
+    for _, pkg_to_find in pairs(package_names) do
+        local pkg_name, pkg_constraint = split_name_constraint(pkg_to_find)
+        pkg_name = utils.escape_magic(pkg_name):gsub("%%%*",".*")
+        for _, repo_pkg in pairs(manifest) do
+            if string.match(repo_pkg.name, "^" .. pkg_name .. "$") and (not pkg_constraint or satisfies_constraint(repo_pkg.version, pkg_constraint)) then
+                table.insert(packages_found, repo_pkg)
+            end
+        end
+    end
+    return packages_found
+end
+
+-- Return manifest consisting of packages installed in specified deploy_dir directory
+function get_installed(deploy_dir)
+    deploy_dir = deploy_dir or cfg.root_dir
+    assert(type(deploy_dir) == "string", "depends.get_installed: Argument 'deploy_dir' is not a string.")
+    deploy_dir = sys.abs_path(deploy_dir)
+
+    local distinfos_path = sys.make_path(deploy_dir, cfg.distinfos_dir)
+    local manifest = {}
+
+    if not sys.is_dir(distinfos_path) then return {} end
+
+    -- from all directories of packages installed in deploy_dir
+    for dir in sys.get_directory(distinfos_path) do
+
+        if dir ~= "." and dir ~= ".." and sys.is_dir(sys.make_path(distinfos_path, dir)) then
+            local pkg_dist_dir = sys.make_path(distinfos_path, dir)
+
+            -- load the dist.info file
+            for file in sys.get_directory(pkg_dist_dir) do
+                local pkg_dist_file = sys.make_path(pkg_dist_dir, file)
+
+                if sys.is_file(pkg_dist_file) then
+                    table.insert(manifest, mf.load_distinfo(pkg_dist_file))
+                end
+            end
+
+        end
+
+    end
+    return manifest
+end
+
+-- If 'pkg.selected' == true then returns 'selected' else 'installed'.
+-- Used in error messages.
+local function selected_or_installed(pkg)
+    assert(type(pkg) == "table", "depends.selected_or_installed: Argument 'pkg' is not a table.")
+    if pkg.selected == true then
+        return "selected"
+    else
+        return "installed"
+    end
+end
+
+-- Return whether the 'package_name' is installed according to the the manifest 'installed_pkgs'
+-- If optional 'version_wanted' constraint is specified, then installed packages must
+-- also satisfy specified version constraint.
+-- If package is installed but doesn't satisfy version constraint, error message
+-- is returned as the second value.
+function is_installed(package_name, installed_pkgs, version_wanted)
+    assert(type(package_name) == "string", "depends.is_installed: Argument 'package_name' is not a string.")
+    assert(type(installed_pkgs) == "table", "depends.is_installed: Argument 'installed_pkgs' is not a table.")
+    assert(type(version_wanted) == "string" or type(version_wanted) == "nil", "depends.is_installed: Argument 'version_wanted' is not a string or nil.")
+
+    local pkg_is_installed, err = false, nil
+
+    for _, installed_pkg in pairs(installed_pkgs) do
+
+        -- check if package_name is in installed
+        if package_name == installed_pkg.name then
+
+            -- check if package is installed in satisfying version
+            if not version_wanted or satisfies_constraint(installed_pkg.version, version_wanted) then
+                pkg_is_installed = true
+                break
+            else
+                err = "Package '" .. package_name .. (version_wanted and " " .. version_wanted or "") .. "' needed, but " .. selected_or_installed(installed_pkg) .. " at version '" .. installed_pkg.version .. "'."
+                break
+            end
+        end
+
+    end
+    return pkg_is_installed, err
+end
+
+-- Check whether the package 'pkg' conflicts with 'installed_pkg' and return
+-- false or error message.
+local function packages_conflicts(pkg, installed_pkg)
+    assert(type(pkg) == "table", "depends.packages_conflicts: Argument 'pkg' is not a table.")
+    assert(type(installed_pkg) == "table", "depends.packages_conflicts: Argument 'installed_pkg' is not a table.")
+
+    -- check if pkg doesn't provide an already installed_pkg
+    if pkg.provides then
+        -- for all of pkg's provides
+        for _, provided_pkg in pairs(get_provides(pkg)) do
+            if provided_pkg.name == installed_pkg.name then
+                return "Package '" .. pkg_full_name(pkg.name, pkg.version, pkg.was_scm_version) .. "' provides '" .. pkg_full_name(provided_pkg.name, provided_pkg.version) .. "' but package '" .. pkg_full_name(installed_pkg.name, installed_pkg.version) .. "' is already " .. selected_or_installed(installed_pkg) .. "."
+            end
+        end
+    end
+
+    -- check for conflicts of package to install with installed package
+    if pkg.conflicts then
+        for _, conflict in pairs (pkg.conflicts) do
+            if conflict == installed_pkg.name then
+                return "Package '" .. pkg_full_name(pkg.name, pkg.version, pkg.was_scm_version) .. "' conflicts with already " .. selected_or_installed(installed_pkg) .. " package '" .. pkg_full_name(installed_pkg.name, installed_pkg.version) .. "'."
+            end
+        end
+    end
+
+    -- check for conflicts of installed package with package to install
+    if installed_pkg.conflicts then
+
+        -- direct conflicts with 'pkg'
+        for _, conflict in pairs (installed_pkg.conflicts) do
+            if conflict == pkg.name then
+                return "Already " .. selected_or_installed(installed_pkg) .. " package '" .. pkg_full_name(installed_pkg.name, installed_pkg.version) .. "' conflicts with package '" .. pkg_full_name(pkg.name, pkg.version, pkg.was_scm_version) .. "'."
+            end
+        end
+
+        -- conflicts with 'provides' of 'pkg' (packages provided by package to install)
+        if pkg.provides then
+            for _, conflict in pairs (installed_pkg.conflicts) do
+                -- for all of pkg's provides
+                for _, provided_pkg in pairs(get_provides(pkg)) do
+                    if conflict == provided_pkg.name then
+                        return "Already '" .. selected_or_installed(installed_pkg) .. " package '" .. pkg_full_name(installed_pkg.name, installed_pkg.version) .. "' conflicts with package '" .. pkg_full_name(provided_pkg.name, provided_pkg.version) .. "' provided by '" .. pkg_full_name(pkg.name, pkg.version, pkg.was_scm_version) .. "'."
+                    end
+                end
+            end
+        end
+    end
+
+    -- no conflicts found
+    return false
+end
+
+-- Return table of package dependencies 'depends' with OS specific dependencies extracted.
+--
+-- OS specific dependencies are stored in a subtable with 'arch' as a key.
+-- E.g. this table containing OS specific dependencies:
+-- depends = {
+--             "lua~>5.1",
+--             "luadist-git>=0.1",
+--             Linux = {
+--                       "iup>=3.6",
+--                       "wxlua>=2.8.10.0",
+--                     },
+--             Windows = {
+--                         "luagd>=2.0.33r2",
+--                         "luacom>=1.4.1",
+--                       },
+--           }
+--
+-- ...will be on the 'Linux' architecture (determined by cfg.arch) converted into:
+-- depends = {
+--             "lua~>5.1",
+--             "luadist-git>=0.1",
+--             "iup>=3.6",
+--             "wxlua>=2.8.10.0",
+--           }
+function extract_os_specific_depends(depends)
+    assert(type(depends) == "table", "depends.extract_os_specific_depends: Argument 'depends' is not a table.")
+    local extracted = {}
+    for k, depend in pairs(depends) do
+        -- if 'depend' is a table, then it must be a table of OS specific
+        -- dependencies, so extract it if it's for this architecture
+        if type(depend) == "table" then
+            if k == cfg.arch then
+                for _, os_specific_depend in pairs(depend) do
+                    table.insert(extracted, os_specific_depend)
+                end
+            end
+        else
+            table.insert(extracted, depend)
+        end
+    end
+    return extracted
+end
+
+-- Return all packages needed in order to install package 'pkg'
+-- and with specified 'installed' packages in the system using 'manifest'.
+-- 'pkg' can also contain version constraint (e.g. 'copas>=1.2.3', 'saci-1.0' etc.).
+--
+-- This function also downloads packages to get information about their dependencies.
+-- Directory where the package was downloaded is stored in 'download_dir' attribute
+-- of that package in the table of packages returned by this function.
+--
+-- Optional argument 'dependency_manifest' is a table of dependencies examined
+-- from previous installations etc. It can be used to speed-up the dependency
+-- resolving procedure for example.
+--
+-- When optional 'force_no_download' parameter is set to true, then information
+-- about packages won't be downloaded during dependency resolving, assuming that
+-- entries in the provided manifest are already complete.
+--
+-- When optional 'suppress_printing' parameter is set to true, then messages
+-- for the user won't be printed during dependency resolving.
+--
+-- Optional argument 'deploy_dir' is used just as a temporary place to place
+-- the downloaded packages into.
+--
+-- 'dependency_parents' is table of all packages encountered so far when resolving dependencies
+-- and is used to detect and deal with circular dependencies. Leave it 'nil'
+-- and it will do its job just fine :-).
+--
+-- 'tmp_installed' is internal table used in recursion and should be left 'nil' when
+-- calling this function from other context. It is used for passing the changes
+-- in installed packages between the recursive calls of this function.
+--
+-- TODO: refactor this spaghetti code!
+local function get_packages_to_install(pkg, installed, manifest, dependency_manifest, force_no_download, suppress_printing, deploy_dir, dependency_parents, tmp_installed)
+    manifest = manifest or mf.get_manifest()
+    dependency_manifest = dependency_manifest or {}
+    force_no_download = force_no_download or false
+    suppress_printing = suppress_printing or false
+    deploy_dir = deploy_dir or cfg.root_dir
+    dependency_parents = dependency_parents or {}
+
+    -- set helper table 'tmp_installed'
+    tmp_installed = tmp_installed or utils.deepcopy(installed)
+
+    assert(type(pkg) == "string", "depends.get_packages_to_install: Argument 'pkg' is not a string.")
+    assert(type(installed) == "table", "depends.get_packages_to_install: Argument 'installed' is not a table.")
+    assert(type(manifest) == "table", "depends.get_packages_to_install: Argument 'manifest' is not a table.")
+    assert(type(dependency_manifest) == "table", "depends.get_packages_to_install: Argument 'dependency_manifest' is not a table.")
+    assert(type(force_no_download) == "boolean", "depends.get_packages_to_install: Argument 'force_no_download' is not a boolean.")
+    assert(type(suppress_printing) == "boolean", "depends.get_packages_to_install: Argument 'suppress_printing' is not a boolean.")
+    assert(type(deploy_dir) == "string", "depends.get_packages_to_install: Argument 'deploy_dir' is not a string.")
+    assert(type(dependency_parents) == "table", "depends.get_packages_to_install: Argument 'dependency_parents' is not a table.")
+    assert(type(tmp_installed) == "table", "depends.get_packages_to_install: Argument 'tmp_installed' is not a table.")
+    deploy_dir = sys.abs_path(deploy_dir)
+
+    --[[ for future debugging:
+    print('resolving: '.. pkg)
+    print('    installed: ', utils.table_tostring(installed))
+    print('    tmp_installed: ', utils.table_tostring(tmp_installed))
+    --]]
+
+    -- check if package is already installed
+    local pkg_name, pkg_constraint = split_name_constraint(pkg)
+    local pkg_is_installed, err = is_installed(pkg_name, tmp_installed, pkg_constraint)
+    if pkg_is_installed then return {} end
+    if err then return nil, err end
+
+    -- table of packages needed to be installed (will be returned)
+    local to_install = {}
+
+    -- find out available versions of 'pkg' and insert them into manifest
+    if not force_no_download then
+        local versions, err = package.retrieve_versions(pkg, manifest, suppress_printing)
+        if not versions then return nil, err end
+        for _, version in pairs(versions) do
+            table.insert(manifest, version)
+        end
+    end
+
+    -- find candidates & sort them
+    local candidates_to_install = find_packages(pkg, manifest)
+    if #candidates_to_install == 0 then
+        return nil, "No suitable candidate for '" .. pkg .. "' found."
+    end
+    candidates_to_install = sort_by_versions(candidates_to_install)
+
+    for _, pkg in pairs(candidates_to_install) do
+
+        --[[ for future debugging:
+        print('  candidate: '.. pkg.name..'-'..pkg.version)
+        print('      installed: ', utils.table_tostring(installed))
+        print('      tmp_installed: ', utils.table_tostring(tmp_installed))
+        print('      to_install: ', utils.table_tostring(to_install))
+        print('  -is installed: ', is_installed(pkg.name, tmp_installed, pkg_constraint))
+        --]]
+
+        -- if there's an error from the previous candidate, print the reason for trying another one
+        if not suppress_printing and err then print(" - trying another candidate due to: " .. err) end
+
+        -- clear the state from the previous candidate
+        pkg_is_installed, err = false, nil
+
+        -- check whether this package has already been added to 'tmp_installed' by another of its candidates
+        pkg_is_installed, err = is_installed(pkg.name, tmp_installed, pkg_constraint)
+        if pkg_is_installed then break end
+
+        -- preserve information about the 'scm' version, because pkg.version
+        -- will be rewritten by information taken from pkg's dist.info file
+        local was_scm_version = (pkg.version == "scm")
+
+        -- Try to obtain cached dependency information from the dependency manifest
+        if dependency_manifest[pkg.name .. "-" .. pkg.version] and cfg.dep_cache then
+            pkg = dependency_manifest[pkg.name .. "-" .. pkg.version]
+        else
+            -- download info about the package if not already downloaded and downloading not prohibited
+            if not (pkg.download_dir or force_no_download) then
+                local path_or_err
+                pkg, path_or_err = package.retrieve_pkg_info(pkg, deploy_dir, suppress_printing)
+                if not pkg then
+                    err = "Error when resolving dependencies: " .. path_or_err
+                else
+                    -- set path to downloaded package - used to indicate that the
+                    -- package was already downloaded, to delete unused but downloaded
+                    -- packages and also to install choosen packages
+                    pkg.download_dir = path_or_err
+                end
+            end
+        end
+
+        if pkg and was_scm_version then pkg.was_scm_version = true end
+
+        -- check arch & type
+        if not err then
+            if not (pkg.arch == "Universal" or pkg.arch == cfg.arch) or
+               not (pkg.type == "all" or pkg.type == "source" or pkg.type == cfg.type) then
+                err = "Package '" .. pkg_full_name(pkg.name, pkg.version) .. "' doesn't have required arch and type."
+            end
+        end
+
+        -- checks for conflicts with other installed (or previously selected) packages
+        if not err then
+            for _, installed_pkg in pairs(tmp_installed) do
+                err = packages_conflicts(pkg, installed_pkg)
+                if err then break end
+            end
+        end
+
+        -- if pkg passed all of the above tests
+        if not err then
+
+            -- check if pkg's dependencies are satisfied
+            if pkg.depends then
+
+                -- insert pkg into the stack of circular dependencies detection
+                table.insert(dependency_parents, pkg.name)
+
+                -- extract all OS specific dependencies of pkg
+                pkg.depends = extract_os_specific_depends(pkg.depends)
+
+                -- for all dependencies of pkg
+                for _, depend in pairs(pkg.depends) do
+                    local dep_name = split_name_constraint(depend)
+
+                    -- detect circular dependencies using 'dependency_parents'
+                    local is_circular_dependency = false
+                    for _, parent in pairs(dependency_parents) do
+                        if dep_name == parent then
+                            is_circular_dependency = true
+                            break
+                        end
+                    end
+
+                    -- if circular dependencies not detected
+                    if not is_circular_dependency then
+
+                        -- recursively call this function on the candidates of this pkg's dependency
+                        local depends_to_install, dep_err = get_packages_to_install(depend, installed, manifest, dependency_manifest, force_no_download, suppress_printing, deploy_dir, dependency_parents, tmp_installed)
+
+                        -- if any suitable dependency packages were found, insert them to the 'to_install' table
+                        if depends_to_install then
+                            for _, depend_to_install in pairs(depends_to_install) do
+
+                                -- add some meta information
+                                if not depend_to_install.selected_by then
+                                    depend_to_install.selected_by = pkg.name .. "-" .. pkg.version
+                                end
+
+                                table.insert(to_install, depend_to_install)
+                                table.insert(tmp_installed, depend_to_install)
+                                table.insert(installed, depend_to_install)
+                            end
+                        else
+                            err = "Error getting dependency of '" .. pkg_full_name(pkg.name, pkg.version) .. "': " .. dep_err
+                            break
+                        end
+
+                    -- if circular dependencies detected
+                    else
+                        err = "Error getting dependency of '" .. pkg_full_name(pkg.name, pkg.version) .. "': '" .. dep_name .. "' is a circular dependency."
+                        break
+                    end
+                end
+
+                -- remove last package from the stack of circular dependencies detection
+                table.remove(dependency_parents)
+            end
+
+            -- if no error occured
+            if not err then
+                -- add pkg and it's provides to the fake table of installed packages, with
+                -- property 'selected' set, indicating that the package isn't
+                -- really installed in the system, just selected to be installed (this is used e.g. in error messages)
+                pkg.selected = true
+                table.insert(tmp_installed, pkg)
+                if pkg.provides then
+                    for _, provided_pkg in pairs(get_provides(pkg)) do
+                        provided_pkg.selected = true
+                        table.insert(tmp_installed, provided_pkg)
+                    end
+                end
+                -- add pkg to the table of packages to install
+                table.insert(to_install, pkg)
+
+            -- if some error occured
+            else
+                -- delete the downloaded package
+                if pkg.download_dir and not cfg.debug then sys.delete(pkg.download_dir) end
+
+                -- set tables of 'packages to install' and 'installed packages' to their original state
+
+                to_install = {}
+                tmp_installed = utils.deepcopy(installed)
+                -- add provided packages to installed ones
+                for _, installed_pkg in pairs(tmp_installed) do
+                    for _, pkg in pairs(get_provides(installed_pkg)) do
+                        table.insert(tmp_installed, pkg)
+                    end
+                end
+            end
+
+        -- if error occured
+        else
+            -- delete the downloaded package
+            if pkg and pkg.download_dir and not cfg.debug then sys.delete(pkg.download_dir) end
+
+            -- if pkg is already installed, skip checking its other candidates
+            if pkg_is_installed then break end
+        end
+    end
+
+    -- if package is not installed and no suitable candidates were found, return the last error
+    if #to_install == 0 and not pkg_is_installed then
+        return nil, err
+    else
+        return to_install
+    end
+end
+
+-- Resolve dependencies and return all packages needed in order to install
+-- 'packages' into the system with already 'installed' packages, using 'manifest'.
+-- Also return the table of the dependencies determined during the process
+-- as the second return value.
+--
+-- Optional argument 'dependency_manifest' is a table of dependencies examined
+-- from previous installations etc. It can be used to speed-up the dependency
+-- resolving procedure for example.
+--
+-- Optional argument 'deploy_dir' is used as a temporary place to place the
+-- downloaded packages into.
+--
+-- When optional 'force_no_download' parameter is set to true, then information
+-- about packages won't be downloaded during dependency resolving, assuming that
+-- entries in manifest are complete.
+--
+-- When optional 'suppress_printing' parameter is set to true, then messages
+-- for the user won't be printed during dependency resolving.
+function get_depends(packages, installed, manifest, dependency_manifest, deploy_dir, force_no_download, suppress_printing)
+    if not packages then return {} end
+    manifest = manifest or mf.get_manifest()
+    dependency_manifest = dependency_manifest or {}
+    deploy_dir = deploy_dir or cfg.root_dir
+    force_no_download = force_no_download or false
+    suppress_printing = suppress_printing or false
+    if type(packages) == "string" then packages = {packages} end
+
+    assert(type(packages) == "table", "depends.get_depends: Argument 'packages' is not a table or string.")
+    assert(type(installed) == "table", "depends.get_depends: Argument 'installed' is not a table.")
+    assert(type(manifest) == "table", "depends.get_depends: Argument 'manifest' is not a table.")
+    assert(type(dependency_manifest) == "table", "depends.get_depends: Argument 'dependency_manifest' is not a table.")
+    assert(type(deploy_dir) == "string", "depends.get_depends: Argument 'deploy_dir' is not a string.")
+    assert(type(force_no_download) == "boolean", "depends.get_depends: Argument 'force_no_download' is not a boolean.")
+    assert(type(suppress_printing) == "boolean", "depends.get_depends: Argument 'suppress_printing' is not a boolean.")
+    deploy_dir = sys.abs_path(deploy_dir)
+
+    local tmp_installed = utils.deepcopy(installed)
+
+    -- add provided packages to installed ones
+    for _, installed_pkg in pairs(tmp_installed) do
+        for _, pkg in pairs(get_provides(installed_pkg)) do
+            table.insert(tmp_installed, pkg)
+        end
+    end
+
+    -- If 'pkg' contains valid (architecture specific) path separator,
+    -- it is treated like a path to already downloaded package and
+    -- we assume that user wants to use this specific version of the
+    -- module to be installed. Hence, we will add information about
+    -- this version into the manifest and also remove references to
+    -- any other versions of this module from the manifest. This will
+    -- enforce the version of the module required by the user.
+    for k, pkg in pairs(packages) do
+        if pkg:find(sys.path_separator()) then
+            local pkg_dir = sys.abs_path(pkg)
+            local pkg_info, err = mf.load_distinfo(sys.make_path(pkg_dir, "dist.info"))
+            if not pkg_info then return nil, err end
+
+            -- add information about location of the package, also to prevent downloading it again
+            pkg_info.download_dir = pkg_dir
+            -- mark package to skip deleting its directory after installation
+            pkg_info.preserve_pkg_dir = true
+
+            -- set default arch/type if not explicitly stated and package is of source type
+            if package.is_source_type(pkg_dir) then
+                pkg_info = package.ensure_source_arch_and_type(pkg_info)
+            elseif not (pkg_info.arch and pkg_info.type) then
+                return nil, pkg_dir .. ": binary package missing arch or type in 'dist.info'."
+            end
+
+            -- update manifest
+            manifest = utils.filter(manifest, function(p) return p.name ~= pkg_info.name and true end)
+            table.insert(manifest, pkg_info)
+
+            -- update packages to install
+            pkg = pkg_info.name .. "-" .. pkg_info.version
+            packages[k] = pkg
+        end
+    end
+
+    local to_install = {}
+
+    -- get packages needed to satisfy the dependencies
+    for _, pkg in pairs(packages) do
+
+        local needed_to_install, err = get_packages_to_install(pkg, tmp_installed, manifest, dependency_manifest, force_no_download, suppress_printing, deploy_dir)
+
+        -- if everything's fine
+        if needed_to_install then
+
+            for _, needed_pkg in pairs(needed_to_install) do
+
+                -- TODO: why not to use 'installed' instead of 'tmp_installed'?
+                --       It's because provides aren't searched for by find()
+                --       function inside the update_dependency_manifest().
+                dependency_manifest = update_dependency_manifest(needed_pkg, tmp_installed, needed_to_install, dependency_manifest)
+
+                table.insert(to_install, needed_pkg)
+                table.insert(tmp_installed, needed_pkg)
+                -- add provides of needed_pkg to installed ones
+                for _, provided_pkg in pairs(get_provides(needed_pkg)) do
+                    -- copy 'selected' property
+                    provided_pkg.selected = needed_pkg.selected
+                    table.insert(tmp_installed, provided_pkg)
+                end
+            end
+        -- if error occured
+        else
+            -- delete already downloaded packages
+            for _, pkg in pairs(to_install) do
+                if pkg.download_dir and not cfg.debug then sys.delete(pkg.download_dir) end
+            end
+            return nil, "Cannot resolve dependencies for '" .. pkg .. "': ".. err
+        end
+    end
+
+    return to_install, dependency_manifest
+end
+
+-- Return table of packages provided by specified package (from it's 'provides' field)
+function get_provides(package)
+    assert(type(package) == "table", "depends.get_provides: Argument 'package' is not a table.")
+    if not package.provides then return {} end
+
+    local provided = {}
+    for _, provided_name in pairs(package.provides) do
+        local pkg = {}
+        pkg.name, pkg.version = split_name_constraint(provided_name)
+        pkg.type = package.type
+        pkg.arch = package.arch
+        pkg.provided = package.name .. "-" .. package.version
+        table.insert(provided, pkg)
+    end
+    return provided
+end
+
+-- Return package name and version constraint from full package version constraint specification
+-- E. g.:
+--          for 'luaexpat-1.2.3'  return:  'luaexpat' , '1.2.3'
+--          for 'luajit >= 1.2'   return:  'luajit'   , '>=1.2'
+function split_name_constraint(version_constraint)
+    assert(type(version_constraint) == "string", "depends.split_name_constraint: Argument 'version_constraint' is not a string.")
+
+    local split = version_constraint:find("[%s=~<>-]+%d") or version_constraint:find("[%s=~<>-]+scm")
+
+    if split then
+        return version_constraint:sub(1, split - 1), version_constraint:sub(split):gsub("[%s-]", "")
+    else
+        return version_constraint, nil
+    end
+end
+
+-- Return only packages that can be installed on the specified architecture and type
+function filter_packages_by_arch_and_type(packages, req_arch, req_type)
+    assert(type(packages) == "table", "depends.filter_packages_by_arch_and_type: Argument 'packages' is not a table.")
+    assert(type(req_arch) == "string", "depends.filter_packages_by_arch_and_type: Argument 'req_arch' is not a string.")
+    assert(type(req_type) == "string", "depends.filter_packages_by_arch_and_type: Argument 'pkg_type' is not a string.")
+
+    return utils.filter(packages,
+                        function (pkg)
+                                return (pkg.arch == "Universal" or pkg.arch == req_arch) and
+                                        (pkg.type == "all" or pkg.type == "source" or pkg.type == req_type)
+                                end)
+end
+
+-- Return only packages that contain one of the specified strings in their 'name-version'.
+-- Case is ignored. If no strings are specified, return all the packages.
+-- Argument 'search_in_desc' specifies if search also in description of packages.
+function filter_packages_by_strings(packages, strings, search_in_desc)
+    if type(strings) == "string" then strings = {strings} end
+    assert(type(packages) == "table", "depends.filter_packages_by_strings: Argument 'packages' is not a table.")
+    assert(type(strings) == "table", "depends.filter_packages_by_strings: Argument 'strings' is not a string or table.")
+
+    if #strings ~= 0 then
+        return utils.filter(packages,
+                            function (pkg)
+                                    for _,str in pairs(strings) do
+                                        local name = pkg.name .. "-" .. pkg.version
+                                        if search_in_desc then
+                                            name = name .. " " .. (pkg.desc or "")
+                                        end
+                                        if string.find(string.lower(name), string.lower(str), 1 ,true) ~= nil then return true end
+                                    end
+                                end)
+    else
+        return packages
+    end
+end
+
+
+-- Return full package name and version string (e.g. 'luajit-2.0'). When version
+-- is nil or '' then return only name (e.g. 'luajit') and when name is nil or ''
+-- then return '<unknown>'. Optional 'was_scm_version' argument is a boolean,
+-- stating whether the package was originally selected for installation as a 'scm' version.
+function pkg_full_name(name, version, was_scm_version)
+    name = name or ""
+    version = version or ""
+    was_scm_version = was_scm_version or false
+    if type(version) == "number" then version = tostring(version) end
+
+    assert(type(name) == "string", "depends.pkg_full_name: Argument 'name' is not a string.")
+    assert(type(version) == "string", "depends.pkg_full_name: Argument 'version' is not a string.")
+
+    if was_scm_version then version = version .. " [scm version]" end
+
+    if name == "" then
+        return "<unknown>"
+    else
+        return name .. ((version ~= "") and "-" .. version or "")
+    end
+end
+
+-- Return table of packages, sorted descendingly by versions (newer ones are moved to the top).
+function sort_by_versions(packages)
+    assert(type(packages) == "table", "depends.sort_by_versions: Argument 'packages' is not a table.")
+    return utils.sort(packages, function (a, b) return compare_versions(a.version, b.version) end)
+end
+
+-- Return table of packages, sorted alphabetically by name and then descendingly by version.
+function sort_by_names(packages)
+    assert(type(packages) == "table", "depends.sort_by_names: Argument 'packages' is not a table.")
+    return utils.sort(packages, function (a, b)
+                        if a.name == b.name then
+                            return compare_versions(a.version, b.version)
+                        else
+                            return a.name < b.name
+                        end
+                     end)
+end
+
+-- Return if version satisfies the specified constraint
+function satisfies_constraint(version, constraint)
+    assert(type(version) == "string", "depends.satisfies_constraint: Argument 'version' is not a string.")
+    assert(type(constraint) == "string", "depends.satisfies_constraint: Argument 'constraint' is not a string.")
+    return const.constraint_satisfied(version, constraint)
+end
+
+-- For package versions, return whether: 'version_a' > 'version_b'
+function compare_versions(version_a, version_b)
+    assert(type(version_a) == "string", "depends.compare_versions: Argument 'version_a' is not a string.")
+    assert(type(version_b) == "string", "depends.compare_versions: Argument 'version_b' is not a string.")
+    return const.compareVersions(version_a, version_b)
+end
+
+-- Returns 'dep_manifest' updated with information about the 'pkg'.
+-- 'installed' is table with installed packages
+-- 'to_install' is table with packages that are selected for installation
+-- Packages satisfying the dependencies will be searched for in these two tables.
+function update_dependency_manifest(pkg, installed, to_install, dep_manifest)
+    dep_manifest = dep_manifest or {}
+    assert(type(pkg) == "table", "depends.update_dependency_manifest: Argument 'pkg' is not a table.")
+    assert(type(installed) == "table", "depends.update_dependency_manifest: Argument 'installed' is not a table.")
+    assert(type(to_install) == "table", "depends.update_dependency_manifest: Argument 'to_install' is not a table.")
+    assert(type(dep_manifest) == "table", "depends.update_dependency_manifest: Argument 'dep_manifest' is not a table.")
+
+    local name_ver = pkg.name .. "-" .. (pkg.was_scm_version  and "scm" or pkg.version)
+
+    -- add to manifest
+    if not dep_manifest[name_ver] then
+        dep_manifest[name_ver] = {}
+        dep_manifest[name_ver].name = pkg.name
+        dep_manifest[name_ver].version = pkg.version
+        dep_manifest[name_ver].was_scm_version = pkg.was_scm_version
+        dep_manifest[name_ver].arch = pkg.arch
+        dep_manifest[name_ver].type = pkg.type
+        dep_manifest[name_ver].path = pkg.path
+        dep_manifest[name_ver].depends = pkg.depends
+        dep_manifest[name_ver].conflicts = pkg.conflicts
+        dep_manifest[name_ver].provides = pkg.provides
+        dep_manifest[name_ver].license = pkg.license
+        dep_manifest[name_ver].desc = pkg.desc
+        dep_manifest[name_ver].url = pkg.url
+        dep_manifest[name_ver].author = pkg.author
+        dep_manifest[name_ver].maintainer = pkg.maintainer
+
+        -- add information which dependency is satisfied by which package
+        if pkg.depends then
+
+            -- TODO: Won't it be better to add OS-specific 'satisfied_by' metadata in a format like OS-specific 'depends' ?
+            local all_deps = extract_os_specific_depends(pkg.depends)
+
+            dep_manifest[name_ver].satisfied_by = {}
+            for _, depend in pairs(all_deps) do
+
+                -- find package satisfying the dependency
+                local satisfying = find_packages(depend, installed)[1] or find_packages(depend, to_install)[1]
+                satisfying = satisfying.name .. "-" .. satisfying.version
+                dep_manifest[name_ver].satisfied_by[depend] = satisfying
+
+                -- check whether the satisfying package isn't provided by other one
+                local provided_by = utils.filter(installed, function(pkg)
+                                                                return pkg.provides and utils.contains(pkg.provides, satisfying)
+                                                            end)
+                if #provided_by == 0 then
+                    provided_by = utils.filter(to_install, function(pkg)
+                                                               return pkg.provides and utils.contains(pkg.provides, satisfying)
+                                                           end)
+                end
+
+                if #provided_by ~= 0 then
+                    if not dep_manifest[name_ver].satisfying_provided_by then
+                        dep_manifest[name_ver].satisfying_provided_by = {}
+                    end
+                    dep_manifest[name_ver].satisfying_provided_by[satisfying] = provided_by[1].name .. "-" .. provided_by[1].version
+                end
+            end
+
+        end
+    end
+
+    return dep_manifest
+end

lualibs/dist/git.lua

@@ -0,0 +1,306 @@
+-- Encapsulated Git functionality
+
+module ("dist.git", package.seeall)
+
+require "git"
+local sys = require "dist.sys"
+local cfg = require "dist.config"
+
+
+-- Clone the repository from url to dest_dir
+function clone(repository_url, dest_dir, depth, branch)
+    assert(type(repository_url) == "string", "git.clone: Argument 'repository_url' is not a string.")
+    assert(type(dest_dir) == "string", "git.clone: Argument 'dest_dir' is not a string.")
+    dest_dir = sys.abs_path(dest_dir)
+
+    local command = "git clone " .. repository_url
+
+    if depth then
+        assert(type(depth) == "number", "git.clone: Argument 'depth' is not a number.")
+        command = command .. " --depth " .. depth
+    end
+
+    if branch then
+        assert(type(branch) == "string", "git.clone: Argument 'branch' is not a string.")
+        command = command .. " -b " .. branch
+    end
+
+    command = command .. " " .. sys.quote(dest_dir)
+    if sys.exists(dest_dir) then sys.delete(dest_dir) end
+    sys.make_dir(dest_dir)
+
+    -- change the current working directory to dest_dir
+    local prev_current_dir = sys.current_dir()
+    sys.change_dir(dest_dir)
+
+    -- execute git clone
+    if not cfg.debug then command = command .. " -q " end
+    local ok, err = sys.exec(command)
+
+    -- change the current working directory back
+    sys.change_dir(prev_current_dir)
+
+    return ok, err
+end
+
+-- Return table of all refs of the remote repository at the 'git_url'. Ref_type can be "tags" or "heads".
+local function get_remote_refs(git_url, ref_type)
+    assert(type(git_url) == "string", "git.get_remote_refs: Argument 'git_url' is not a string.")
+    assert(type(ref_type) == "string", "git.get_remote_refs: Argument 'ref_type' is not a string.")
+    assert(ref_type == "tags" or ref_type == "heads", "git.get_remote_refs: Argument 'ref_type' is not \"tags\" or \"heads\".")
+
+    local refs = {}
+
+    local ok, refs_or_err = pcall(git.protocol.remotes, git_url)
+    if not ok then return nil, "Error getting refs of the remote repository '" .. git_url .. "': " .. refs_or_err end
+
+    for ref, sha in pairs(refs_or_err) do
+        if ref:match("%S+/" .. ref_type .. "/%S+") and not ref:match("%^{}") then
+            table.insert(refs, ref:match("%S+/" .. ref_type .. "/(%S+)"))
+        end
+    end
+
+    return refs
+end
+
+-- Return table of all tags of the repository at the 'git_url'
+function get_remote_tags(git_url)
+    return get_remote_refs(git_url, "tags")
+end
+
+-- Return table of all branches of the repository at the 'git_url'
+function get_remote_branches(git_url)
+    return get_remote_refs(git_url, "heads")
+end
+
+-- Checkout specified ref in specified git_repo_dir
+function checkout_ref(ref, git_repo_dir, orphaned)
+    git_repo_dir = git_repo_dir or sys.current_dir()
+    orphaned = orphaned or false
+    assert(type(ref) == "string", "git.checkout_ref: Argument 'ref' is not a string.")
+    assert(type(git_repo_dir) == "string", "git.checkout_ref: Argument 'git_repo_dir' is not a string.")
+    assert(type(orphaned) == "boolean", "git.checkout_ref: Argument 'orphaned' is not a boolean.")
+    git_repo_dir = sys.abs_path(git_repo_dir)
+
+    local command = "git checkout "
+    if orphaned then command = command .. " --orphan " end
+    command = command .. " " .. ref .. " -f"
+    if not cfg.debug then command = command .. " -q " end
+
+    local ok, err
+    if git_repo_dir ~= sys.current_dir() then
+        local prev_current_dir = sys.current_dir()
+        sys.change_dir(git_repo_dir)
+        ok, err = sys.exec(command)
+        sys.change_dir(prev_current_dir)
+    else
+        ok, err = sys.exec(command)
+    end
+
+    return ok, err
+end
+
+-- Checkout specified sha in specified git_repo_dir
+function checkout_sha(sha, git_repo_dir)
+    git_repo_dir = git_repo_dir or sys.current_dir()
+    assert(type(sha) == "string", "git.checkout_sha: Argument 'sha' is not a string.")
+    assert(type(git_repo_dir) == "string", "git.checkout_sha: Argument 'git_repo_dir' is not a string.")
+    git_repo_dir = sys.abs_path(git_repo_dir)
+
+    local dir_changed, prev_current_dir
+
+    if git_repo_dir ~= sys.current_dir() then
+        prev_current_dir = sys.current_dir()
+        sys.change_dir(git_repo_dir)
+        dir_changed = true
+    end
+
+    local ok, repo_or_err = pcall(git.repo.open, git_repo_dir)
+    if not ok then return nil, "Error when opening the git repository '" .. git_repo_dir .. "': " .. repo_or_err end
+
+    local err
+    ok, err = pcall(repo_or_err.checkout, repo_or_err, sha, git_repo_dir)
+    if not ok then return nil, "Error when checking out the sha '" .. sha .. "' in the git repository '" .. git_repo_dir .. "': " .. err end
+
+    repo_or_err:close()
+    if dir_changed then sys.change_dir(prev_current_dir) end
+
+    return true
+end
+
+-- Create an empty git repository in given directory.
+function init(dir)
+    dir = dir or sys.current_dir()
+    assert(type(dir) == "string", "git.init: Argument 'dir' is not a string.")
+    dir = sys.abs_path(dir)
+
+    -- create the 'dir' first, since it causes 'git init' to fail on Windows
+    -- when the parent directory of 'dir' doesn't exist
+    local ok, err = sys.make_dir(dir)
+    if not ok then return nil, err end
+
+    local command = "git init " .. sys.quote(dir)
+    if not cfg.debug then command = command .. " -q " end
+    return sys.exec(command)
+end
+
+-- Add all files in the 'repo_dir' to the git index. The 'repo_dir' must be
+-- in the initialized git repository.
+function add_all(repo_dir)
+    repo_dir = repo_dir or sys.current_dir()
+    assert(type(repo_dir) == "string", "git.add_all: Argument 'repo_dir' is not a string.")
+    repo_dir = sys.abs_path(repo_dir)
+
+    local ok, prev_dir, msg
+    ok, prev_dir = sys.change_dir(repo_dir);
+    if not ok then return nil, err end
+
+    ok, msg = sys.exec("git add -A -f " .. sys.quote(repo_dir))
+    sys.change_dir(prev_dir)
+
+    return ok, msg
+end
+
+-- Commit all indexed files in 'repo_dir' with the given commit 'message'.
+-- The 'repo_dir' must be in the initialized git repository.
+function commit(message, repo_dir)
+    repo_dir = repo_dir or sys.current_dir()
+    message = message or "commit by luadist-git"
+    assert(type(message) == "string", "git.commit: Argument 'message' is not a string.")
+    assert(type(repo_dir) == "string", "git.commit: Argument 'repo_dir' is not a string.")
+    repo_dir = sys.abs_path(repo_dir)
+
+    local ok, prev_dir, msg
+    ok, prev_dir = sys.change_dir(repo_dir);
+    if not ok then return nil, err end
+
+    local command = "git commit -m " .. sys.quote(message)
+    if not cfg.debug then command = command .. " -q " end
+    ok, msg = sys.exec(command)
+    sys.change_dir(prev_dir)
+
+    return ok, msg
+end
+
+
+-- Rename branch 'old_name' to 'new_name'. -- The 'repo_dir' must be
+-- in the initialized git repository and the branch 'new_name' must
+-- not already exist in that repository.
+function rename_branch(old_name, new_name, repo_dir)
+    repo_dir = repo_dir or sys.current_dir()
+    assert(type(old_name) == "string", "git.rename_branch: Argument 'old_name' is not a string.")
+    assert(type(new_name) == "string", "git.rename_branch: Argument 'new_name' is not a string.")
+    assert(type(repo_dir) == "string", "git.rename_branch: Argument 'repo_dir' is not a string.")
+    repo_dir = sys.abs_path(repo_dir)
+
+    local ok, prev_dir, msg
+    ok, prev_dir = sys.change_dir(repo_dir);
+    if not ok then return nil, err end
+
+    ok, msg = sys.exec("git branch -m " .. old_name .. " " .. new_name)
+    sys.change_dir(prev_dir)
+
+    return ok, msg
+end
+
+-- Push the ref 'ref_name' from the 'repo_dir' to the remote git
+-- repository 'git_repo_url'. If 'all_tags' is set to true, all tags
+-- will be pushed, in addition to the explicitly given ref.
+-- If 'delete' is set to 'true' then the explicitly given remote ref
+-- will be deleted, not pushed.
+function push_ref(repo_dir, ref_name, git_repo_url, all_tags, delete)
+    repo_dir = repo_dir or sys.current_dir()
+    all_tags = all_tags or false
+    delete = delete or false
+    assert(type(repo_dir) == "string", "git.push_ref: Argument 'repo_dir' is not a string.")
+    assert(type(git_repo_url) == "string", "git.push_ref: Argument 'git_repo_url' is not a string.")
+    assert(type(ref_name) == "string", "git.push_ref: Argument 'ref_name' is not a string.")
+    assert(type(all_tags) == "boolean", "git.push_ref: Argument 'all_tags' is not a boolean.")
+    assert(type(delete) == "boolean", "git.push_ref: Argument 'delete' is not a boolean.")
+    repo_dir = sys.abs_path(repo_dir)
+
+    local ok, prev_dir, msg
+    ok, prev_dir = sys.change_dir(repo_dir);
+    if not ok then return nil, err end
+
+    local command = "git push " .. git_repo_url
+    if all_tags then command = command .. " --tags " end
+    if delete then command = command .. " --delete " end
+    command = command .. " " .. ref_name .. " -f "
+    if not cfg.debug then command = command .. " -q " end
+
+    ok, msg = sys.exec(command)
+    sys.change_dir(prev_dir)
+
+    return ok, msg
+end
+
+-- Creates the tag 'tag_name' in given 'repo_dir', which must be
+-- in the initialized git repository
+function create_tag(repo_dir, tag_name)
+    repo_dir = repo_dir or sys.current_dir()
+    assert(type(repo_dir) == "string", "git.create_tag: Argument 'repo_dir' is not a string.")
+    assert(type(tag_name) == "string", "git.create_tag: Argument 'tag_name' is not a string.")
+    repo_dir = sys.abs_path(repo_dir)
+
+    local ok, prev_dir, msg
+    ok, prev_dir = sys.change_dir(repo_dir);
+    if not ok then return nil, err end
+
+    ok, msg = sys.exec("git tag " .. tag_name .. " -f ")
+    sys.change_dir(prev_dir)
+
+    return ok, msg
+end
+
+-- Fetch given 'ref_name' from the remote 'git_repo_url' to the local repository
+-- 'repo_dir' and return its sha. 'ref_type' can be "tag" or "head".
+local function fetch_ref(repo_dir, git_repo_url, ref_name, ref_type)
+    repo_dir = repo_dir or sys.current_dir()
+    assert(type(repo_dir) == "string", "git.fetch_ref: Argument 'repo_dir' is not a string.")
+    assert(type(git_repo_url) == "string", "git.fetch_ref: Argument 'git_repo_url' is not a string.")
+    assert(type(ref_name) == "string", "git.fetch_ref: Argument 'ref_name' is not a string.")
+    assert(type(ref_type) == "string", "git.fetch_ref: Argument 'ref_type' is not a string.")
+    assert(ref_type == "tag" or ref_type == "head", "git.get_remote_refs: Argument 'ref_type' is not \"tag\" or \"head\".")
+    repo_dir = sys.abs_path(repo_dir)
+
+    local refstring = "refs/" .. ref_type .. "s/" .. ref_name
+
+    local suppress_fetch_progress = not cfg.debug
+    local ok, repo_or_err = pcall(git.repo.open, repo_dir)
+    if not ok then return nil, "Error when opening the git repository '" .. repo_dir .. "': " .. repo_or_err end
+
+    local ok, pack_or_err, sha = pcall(git.protocol.fetch, git_repo_url, repo_or_err, refstring, suppress_fetch_progress)
+    if not ok then return nil, "Error when fetching ref '" .. refstring .. "' from git repository '" .. git_repo_url .. "': " .. pack_or_err end
+
+    repo_or_err:close()
+    pack_or_err:close()
+
+    return sha
+end
+
+-- Fetch given 'tag_name' from the remote 'git_repo_url' to the local repository
+-- 'repo_dir' and save it as a tag with the same 'tag_name'.
+function fetch_tag(repo_dir, git_repo_url, tag_name)
+    return fetch_ref(repo_dir, git_repo_url, tag_name, "tag")
+end
+
+-- Fetch given 'branch_name' from the remote 'git_repo_url' to the local repository
+-- 'repo_dir' and save it as a branch with the same 'branch_name'.
+function fetch_branch(repo_dir, git_repo_url, branch_name)
+    return fetch_ref(repo_dir, git_repo_url, branch_name, "head")
+end
+
+-- Create the git repository and return the repo object (which can be used in checkout_sha etc.)
+-- If the 'dir' exists, it's deleted prior to creating the git repository.
+function create_repo(dir)
+    assert(type(dir) == "string", "git.create_repo: Argument 'dir' is not a string.")
+
+    if sys.exists(dir) then sys.delete(dir) end
+
+    local ok, repo_or_err = pcall(git.repo.create, dir)
+    if not ok then return nil, "Error when creating the git repository '" .. dir .. "': " .. repo_or_err end
+
+    repo_or_err:close()
+    return true
+end

lualibs/dist/init.lua

@@ -0,0 +1,349 @@
+-- main API of LuaDist
+
+module ("dist", package.seeall)
+
+local cfg = require "dist.config"
+local depends = require "dist.depends"
+local git = require "dist.git"
+local sys = require "dist.sys"
+local package = require "dist.package"
+local mf = require "dist.manifest"
+local utils = require "dist.utils"
+
+-- Return the deployment directory.
+function get_deploy_dir()
+    return sys.abs_path(cfg.root_dir)
+end
+
+-- Return packages deployed in 'deploy_dir' also with their provides.
+function get_deployed(deploy_dir)
+    deploy_dir = deploy_dir or cfg.root_dir
+    assert(type(deploy_dir) == "string", "dist.get_deployed: Argument 'deploy_dir' is not a string.")
+    deploy_dir = sys.abs_path(deploy_dir)
+
+    local deployed = depends.get_installed(deploy_dir)
+    local provided = {}
+
+    for _, pkg in pairs(deployed) do
+        for _, provided_pkg in pairs(depends.get_provides(pkg)) do
+            provided_pkg.provided_by = pkg.name .. "-" .. pkg.version
+            table.insert(provided, provided_pkg)
+        end
+    end
+
+    for _, provided_pkg in pairs(provided) do
+        table.insert(deployed, provided_pkg)
+    end
+
+    deployed = depends.sort_by_names(deployed)
+    return deployed
+end
+
+-- Download new 'manifest_file' from repository and returns it.
+-- Return nil and error message on error.
+function update_manifest(deploy_dir)
+    deploy_dir = deploy_dir or cfg.root_dir
+    assert(type(deploy_dir) == "string", "dist.update_manifest: Argument 'deploy_dir' is not a string.")
+    deploy_dir = sys.abs_path(deploy_dir)
+
+    -- TODO: use 'deploy_dir' argument in manifest functions
+
+    -- retrieve the new manifest (forcing no cache use)
+    local manifest, err = mf.get_manifest(nil, true)
+
+    if manifest then
+        return manifest
+    else
+        return nil, err
+    end
+end
+
+-- Install 'package_names' to 'deploy_dir', using optional CMake 'variables'.
+function install(package_names, deploy_dir, variables)
+    if not package_names then return true end
+    deploy_dir = deploy_dir or cfg.root_dir
+    if type(package_names) == "string" then package_names = {package_names} end
+
+    assert(type(package_names) == "table", "dist.install: Argument 'package_names' is not a table or string.")
+    assert(type(deploy_dir) == "string", "dist.install: Argument 'deploy_dir' is not a string.")
+    deploy_dir = sys.abs_path(deploy_dir)
+
+    -- find installed packages
+    local installed = depends.get_installed(deploy_dir)
+
+    -- get manifest
+    local manifest, err = mf.get_manifest()
+    if not manifest then return nil, "Error getting manifest: " .. err end
+
+    -- get dependency manifest
+    -- TODO: Is it good that dep_manifest is deploy_dir-specific?
+    -- Probably it'd be better not to be specific, but then there're
+    -- problems with 'provides'. E.g. What to do if there's a module
+    -- installed, that is provided by two different modules in two deploy_dirs?
+    local dep_manifest_file = sys.abs_path(sys.make_path(deploy_dir, cfg.dep_cache_file))
+    local dep_manifest, status = {}
+    if sys.exists(dep_manifest_file) and not utils.cache_timeout_expired(cfg.cache_timeout, dep_manifest_file) then
+        status, dep_manifest = mf.load_manifest(dep_manifest_file)
+        if not dep_manifest then return nil, status end
+    end
+
+    -- resolve dependencies
+    local dependencies, dep_manifest_or_err = depends.get_depends(package_names, installed, manifest, dep_manifest, deploy_dir, false, false)
+    if not dependencies then return nil, dep_manifest_or_err end
+    if #dependencies == 0 then return nil, "No packages to install." end
+
+    -- save updated dependency manifest
+    local ok, err = sys.make_dir(sys.parent_dir(dep_manifest_file))
+    if not ok then return nil, err end
+    ok, err = mf.save_manifest(dep_manifest_or_err, dep_manifest_file)
+    if not ok then return nil, err end
+
+    -- fetch the packages from repository
+    local fetched_pkgs = {}
+    for _, pkg in pairs(dependencies) do
+        local fetched_pkg, err = package.fetch_pkg(pkg, sys.make_path(deploy_dir, cfg.temp_dir))
+        if not fetched_pkg then return nil, err end
+        table.insert(fetched_pkgs, fetched_pkg)
+    end
+
+    -- install fetched packages
+    for _, pkg in pairs(fetched_pkgs) do
+        local ok, err = package.install_pkg(pkg.download_dir, deploy_dir, variables, pkg.preserve_pkg_dir)
+        if not ok then return nil, err end
+    end
+
+    return true
+end
+
+-- Manually deploy packages from 'package_dirs' to 'deploy_dir', using optional
+-- CMake 'variables'. The 'package_dirs' are preserved (will not be deleted).
+function make(deploy_dir, package_dirs, variables)
+    deploy_dir = deploy_dir or cfg.root_dir
+    package_dirs = package_dirs or {}
+
+    assert(type(deploy_dir) == "string", "dist.make: Argument 'deploy_dir' is not a string.")
+    assert(type(package_dirs) == "table", "dist.make: Argument 'package_dirs' is not a table.")
+    deploy_dir = sys.abs_path(deploy_dir)
+
+    for _, dir in pairs(package_dirs) do
+        local ok, err = package.install_pkg(sys.abs_path(dir), deploy_dir, variables, true)
+        if not ok then return nil, err end
+    end
+    return true
+end
+
+-- Remove 'package_names' from 'deploy_dir' and return the number of removed
+-- packages.
+function remove(package_names, deploy_dir)
+    deploy_dir = deploy_dir or cfg.root_dir
+    if type(package_names) == "string" then package_names = {package_names} end
+
+    assert(type(package_names) == "table", "dist.remove: Argument 'package_names' is not a string or table.")
+    assert(type(deploy_dir) == "string", "dist.remove: Argument 'deploy_dir' is not a string.")
+    deploy_dir = sys.abs_path(deploy_dir)
+
+    local pkgs_to_remove = {}
+    local installed = depends.get_installed(deploy_dir)
+
+    -- find packages to remove
+    if #package_names == 0 then
+        pkgs_to_remove = installed
+    else
+        pkgs_to_remove = depends.find_packages(package_names, installed)
+    end
+
+    -- remove them
+    for _, pkg in pairs(pkgs_to_remove) do
+        local pkg_distinfo_dir = sys.make_path(cfg.distinfos_dir, pkg.name .. "-" .. pkg.version)
+        local ok, err = package.remove_pkg(pkg_distinfo_dir, deploy_dir)
+        if not ok then return nil, err end
+    end
+
+    return #pkgs_to_remove
+end
+
+-- Download 'pkg_names' to 'fetch_dir' and return the table of their directories.
+function fetch(pkg_names, fetch_dir)
+    fetch_dir = fetch_dir or sys.current_dir()
+    assert(type(pkg_names) == "table", "dist.fetch: Argument 'pkg_names' is not a string or table.")
+    assert(type(fetch_dir) == "string", "dist.fetch: Argument 'fetch_dir' is not a string.")
+    fetch_dir = sys.abs_path(fetch_dir)
+
+    local manifest = mf.get_manifest()
+
+    local pkgs_to_fetch = {}
+    for _, pkg_name in pairs(pkg_names) do
+
+        -- retrieve available versions
+        local versions, err = package.retrieve_versions(pkg_name, manifest)
+        if not versions then return nil, err end
+        for _, version in pairs(versions) do
+            table.insert(manifest, version)
+        end
+
+        local packages = depends.find_packages(pkg_name, manifest)
+        if #packages == 0 then return nil, "No packages found for '" .. pkg_name .. "'." end
+
+        packages = depends.sort_by_versions(packages)
+        table.insert(pkgs_to_fetch, packages[1])
+    end
+
+    local fetched_dirs = {}
+    for _, pkg in pairs(pkgs_to_fetch) do
+        local fetched_pkg, err = package.fetch_pkg(pkg, fetch_dir)
+        if not fetched_pkg then return nil, err end
+        table.insert(fetched_dirs, fetched_pkg.download_dir)
+    end
+
+    return fetched_dirs
+end
+
+-- Upload binary version of given modules installed in the specified
+-- 'deploy_dir' to the repository specified by provided base url.
+-- Return the number of uploaded packages.
+--
+-- Organization of uploaded modules and their repositories is subject
+-- to the following conventions:
+--   - destination repository is: 'DEST_GIT_BASE_URL/MODULE_NAME'
+--   - module will be uploaded to the branch: 'ARCH-TYPE' according
+--     to the arch and type of the user's machine
+--   - the module will be tagged as: 'VERSION-ARCH-TYPE' (if the tag already
+--     exists, it will be overwritten)
+--
+-- E.g. assume that the module 'lua-5.1.4' is installed on the 32bit Linux
+-- system (Linux-i686). When this function is called with the module name
+-- 'lua' and base url 'git@github.com:LuaDist', then the binary version
+-- of the module 'lua', that is installed on the machine, will be uploaded
+-- to the branch 'Linux-i686' of the repository 'git@github.com:LuaDist/lua.git'
+-- and tagged as '5.1.4-Linux-i686'.
+function upload_modules(deploy_dir, module_names, dest_git_base_url)
+    deploy_dir = deploy_dir or cfg.root_dir
+    if type(module_names) == "string" then module_names = {module_names} end
+    assert(type(deploy_dir) == "string", "dist.upload_module: Argument 'deploy_dir' is not a string.")
+    assert(type(module_names) == "table", "dist.upload_module: Argument 'module_name' is not a string or table.")
+    assert(type(dest_git_base_url) == "string", "dist.upload_module: Argument 'dest_git_base_url' is not a string.")
+    deploy_dir = sys.abs_path(deploy_dir)
+
+    local modules_to_upload = {}
+    local installed = depends.get_installed(deploy_dir)
+
+    -- find modules to upload
+    if #module_names == 0 then
+        modules_to_upload = installed
+    else
+        modules_to_upload = depends.find_packages(module_names, installed)
+    end
+
+    for _, installed_module in pairs(modules_to_upload) do
+
+        -- set names
+        local branch_name = cfg.arch .. "-" .. cfg.type
+        local tag_name = installed_module.version .. "-" .. branch_name
+        local full_name = installed_module.name .. "-" .. tag_name
+        local tmp_dir = sys.make_path(deploy_dir, cfg.temp_dir, full_name .. "-to-upload")
+        local dest_git_url = dest_git_base_url .. "/" .. installed_module.name .. ".git"
+        local distinfo_file = sys.make_path(deploy_dir, cfg.distinfos_dir, installed_module.name .. "-" .. installed_module.version, "dist.info")
+
+        -- create temporary directory (delete previous if already exists)
+        if sys.exists(tmp_dir) then sys.delete(tmp_dir) end
+        local ok, err = sys.make_dir(tmp_dir)
+        if not ok then return nil, err end
+
+        -- copy the module files for all enabled components
+        for _, component in ipairs(cfg.components) do
+            if installed_module.files[component] then
+                for _, file in ipairs(installed_module.files[component]) do
+                    local file_path = sys.make_path(deploy_dir, file)
+                    local dest_dir = sys.parent_dir(sys.make_path(tmp_dir, file))
+                    if sys.is_file(file_path) then
+                        sys.make_dir(dest_dir)
+                        sys.copy(file_path, dest_dir)
+                    end
+                end
+            end
+        end
+
+        -- add module's dist.info file
+        sys.copy(distinfo_file, tmp_dir)
+
+        -- create git repo
+        ok, err = git.init(tmp_dir)
+        if not ok then return nil, "Error initializing empty git repository in '" .. tmp_dir .. "': " .. err end
+
+        -- add all files
+        ok, err = git.add_all(tmp_dir)
+        if not ok then return nil, "Error adding all files to the git index in '" .. tmp_dir .. "': " .. err end
+
+        -- create commit
+        ok, err = git.commit("[luadist-git] add " .. full_name .. " [ci skip]", tmp_dir)
+        if not ok then return nil, "Error commiting changes in '" .. tmp_dir .. "': " .. err end
+
+        -- rename branch
+        ok, err = git.rename_branch("master", branch_name, tmp_dir)
+        if not ok then return nil, "Error renaming branch 'master' to '" .. branch_name .. "' in '" .. tmp_dir .. "': " .. err  end
+
+        -- create tag
+        ok, err = git.create_tag(tmp_dir, tag_name)
+        if not ok then return nil, "Error creating tag '" .. tag_name .. "' in '" .. tmp_dir .. "': " .. err end
+
+        print("Uploading " .. full_name .. " to " .. dest_git_url .. "...")
+
+        -- push to the repository
+        ok, err = git.push_ref(tmp_dir, branch_name, dest_git_url, true)
+        if not ok then return nil, "Error when pushing branch '" .. branch_name .. "' and tag '" .. tag_name .. "' to '" .. dest_git_url .. "': " .. err end
+
+        -- delete temporary directory (if not in debug mode)
+        if not cfg.debug then sys.delete(tmp_dir) end
+    end
+
+    return #modules_to_upload
+end
+
+-- Returns table with information about module's dependencies, using the cache.
+function dependency_info(module, deploy_dir)
+    cache_file = cache_file or sys.abs_path(sys.make_path(cfg.root_dir, cfg.dep_cache_file))
+    assert(type(module) == "string", "dist.dependency_info: Argument 'module' is not a string.")
+    assert(type(deploy_dir) == "string", "dist.dependency_info: Argument 'deploy_dir' is not a string.")
+
+    -- get manifest
+    local manifest, err = mf.get_manifest()
+    if not manifest then return nil, "Error getting manifest: " .. err end
+
+    -- get dependency manifest
+    -- TODO: Is it good that dep_manifest is deploy_dir-specific?
+    -- Probably it'd be better not to be specific, but then there're
+    -- problems with 'provides'. E.g. What to do if there's a module
+    -- installed, that is provided by two different modules in two deploy_dirs?
+    local dep_manifest_file = sys.abs_path(sys.make_path(deploy_dir, cfg.dep_cache_file))
+    local dep_manifest, status = {}
+    if sys.exists(dep_manifest_file) and cfg.cache and not utils.cache_timeout_expired(cfg.cache_timeout, dep_manifest_file) then
+        status, dep_manifest = mf.load_manifest(dep_manifest_file)
+        if not dep_manifest then return nil, status end
+    end
+
+    -- force getting the dependency information
+    local installed = {}
+
+    -- resolve dependencies
+    local dependencies, dep_manifest_or_err = depends.get_depends(module, installed, manifest, dep_manifest, deploy_dir, false, true and not cfg.debug)
+    if not dependencies then return nil, dep_manifest_or_err end
+
+    -- save updated dependency manifest
+    local ok, err = sys.make_dir(sys.parent_dir(dep_manifest_file))
+    if not ok then return nil, err end
+    ok, err = mf.save_manifest(dep_manifest_or_err, dep_manifest_file)
+    if not ok then return nil, err end
+
+    -- collect just relevant dependencies from dependency manifest
+    local relevant_deps = {}
+    for _, dep in pairs(dependencies) do
+        local name_ver = dep.name .. "-" .. (dep.was_scm_version and "scm" or dep.version)
+        if dep_manifest_or_err[name_ver] then
+            table.insert(relevant_deps, dep_manifest_or_err[name_ver])
+        else
+            return nil, "Error: dependency information for '" .. name_ver .. "' not found in dependency manifest."
+        end
+    end
+
+    return relevant_deps
+end

lualibs/dist/logger.lua

@@ -0,0 +1,64 @@
+-- Simple logger for LuaDist.
+
+module ("dist.logger", package.seeall)
+
+local cfg = require "dist.config"
+local sys = require "dist.sys"
+
+-- Open 'log_file' and return a log, or nil and error msg on error.
+local function get_log(log_file)
+    log_file = log_file or cfg.log_file
+    assert(type(log_file) == "string", "log.get_log: Argument 'log_file' is not a string.")
+    log_file = sys.abs_path(log_file)
+
+    sys.make_dir(sys.parent_dir(log_file))
+    local log, err = io.open(log_file, "a")
+    if not log then
+        return nil, "Error: can't open a logfile '" .. log_file .. "': " .. err
+    else
+        return log
+    end
+end
+
+-- Set the default log.
+local log_file = get_log(cfg.log_file)
+
+-- Log levels used.
+local log_levels = {
+    DEBUG = 0, -- Fine-grained informational events that are most useful to debug an application.
+    INFO  = 1, -- Informational messages that highlight the progress of the application at coarse-grained level.
+    WARN  = 2, -- Potentially harmful situations.
+    ERROR = 3, -- Error events that might still allow the application to continue running.
+    FATAL = 4, -- Very severe error events that would presumably lead the application to abort.
+}
+
+-- Write 'message' with 'level' to 'log'.
+local function write(level, ...)
+    assert(type(level) == "string", "log.write: Argument 'level' is not a string.")
+    assert(#arg > 0, "log.write: No message arguments provided.")
+    assert(type(log_levels[level]) == "number", "log.write: Unknown log level used: '" .. level .. "'.")
+
+    level = level:upper()
+    local message = table.concat(arg, " ")
+
+    -- Check if writing for this log level is enabled.
+    if cfg.write_log_level and log_levels[level] >= log_levels[cfg.write_log_level] then
+        log_file:write(os.date("%Y-%m-%d %H:%M:%S") .. " [" .. level .. "]\t" .. message .. "\n")
+        log_file:flush()
+    end
+
+    -- Check if printing for this log level is enabled.
+    if cfg.print_log_level and log_levels[level] >= log_levels[cfg.print_log_level] then
+        print(message)
+    end
+end
+
+-- Functions with defined log levels for simple use.
+function debug(...) return write("DEBUG", ...) end
+function info(...) return write("INFO", ...) end
+function warn(...) return write("WARN", ...) end
+function error(...) return write("ERROR", ...) end
+function fatal(...) return write("FATAL", ...) end
+
+-- Function with explicitly specified log level.
+function log(level, ...) return write(level, ...) end

lualibs/dist/manifest.lua

@@ -0,0 +1,248 @@
+-- Working with manifest and dist.info files
+
+module ("dist.manifest", package.seeall)
+
+local cfg = require "dist.config"
+local git = require "dist.git"
+local sys = require "dist.sys"
+local utils = require "dist.utils"
+
+-- Return the manifest table from 'manifest_file'. If the manifest is in cache,
+-- then the cached version is used. You can set the cache timeout value in
+-- 'config.cache_timeout' variable.
+-- If optional 'force_no_cache' parameter is true, then the cache is not used.
+function get_manifest(manifest_file, force_no_cache)
+    manifest_file = manifest_file or sys.make_path(cfg.root_dir, cfg.manifest_file)
+    force_no_cache = force_no_cache or false
+
+    assert(type(manifest_file) == "string", "manifest.get_manifest: Argument 'manifest_file' is not a string.")
+    assert(type(force_no_cache) == "boolean", "manifest.get_manifest: Argument 'force_no_cache' is not a boolean.")
+    manifest_file = sys.abs_path(manifest_file)
+
+    -- download new manifest to the cache if not present or cache not used or cache expired
+    if not sys.exists(manifest_file) or force_no_cache or not cfg.cache or utils.cache_timeout_expired(cfg.cache_timeout, manifest_file) then
+        local manifest_dest = sys.parent_dir(manifest_file) or sys.current_dir()
+        local ok, err = download_manifest(manifest_dest, cfg.repos)
+        if not ok then return nil, "Error when downloading manifest: " .. err end
+    end
+
+    -- load manifest from cache
+    local status, ret = load_manifest(manifest_file)
+    if not status then return nil, "Error when loading manifest: " .. ret end
+
+    return ret
+end
+
+-- Download manifest from the table of git 'repository_urls' to 'dest_dir' and return true on success
+-- and nil and error message on error.
+function download_manifest(dest_dir, repository_urls)
+    dest_dir = dest_dir or sys.make_path(cfg.root_dir, cfg.cache_dir)
+    repository_urls = repository_urls or cfg.repos
+    if type(repository_urls) == "string" then repository_urls = {repository_urls} end
+
+    assert(type(dest_dir) == "string", "manifest.download_manifest: Argument 'dest_dir' is not a string.")
+    assert(type(repository_urls) == "table", "manifest.download_manifest: Argument 'repository_urls' is not a table or string.")
+    dest_dir = sys.abs_path(dest_dir)
+
+    -- define used files and directories
+    local manifest_filename = sys.extract_name(cfg.manifest_file)
+    local manifest_file = sys.make_path(dest_dir, manifest_filename)
+    local temp_dir = sys.make_path(cfg.root_dir, cfg.temp_dir)
+
+    -- ensure that destination directory exists
+    local ok, err = sys.make_dir(dest_dir)
+    if not ok then return nil, err end
+
+    -- retrieve manifests from repositories and collect them into one manifest table
+    local manifest = {}
+
+    if #repository_urls == 0 then return nil, "No repository url specified." end
+
+    print("Downloading repository information...")
+    for k, repo in pairs(repository_urls) do
+        local clone_dir = sys.make_path(temp_dir, "repository_" .. tostring(k))
+
+        -- clone the repo and add its '.gitmodules' file to the manifest table
+
+        ok, err = git.create_repo(clone_dir)
+
+        local sha
+        if ok then sha, err = git.fetch_branch(clone_dir, repo, "master") end
+        if sha then ok, err = git.checkout_sha(sha, clone_dir) end
+
+        if not (ok and sha) then
+            if not cfg.debug then sys.delete(clone_dir) end
+            return nil, "Error when downloading the manifest from repository with url: '" .. repo .. "': " .. err
+        else
+            for _, pkg in pairs(load_gitmodules(sys.make_path(clone_dir, ".gitmodules"))) do
+                table.insert(manifest, pkg)
+            end
+        end
+        if not cfg.debug then sys.delete(clone_dir) end
+    end
+
+    -- save the new manifest table to the file
+    ok, err = save_manifest(manifest, manifest_file)
+    if not ok then return nil, err end
+
+    return true
+end
+
+-- A secure loadfile function
+-- If file code chunk has upvalues, the first upvalue is set to the given
+-- environement, if that parameter is given, or to the value of the global environment.
+local function secure_loadfile(file, env)
+    assert(type(file) == "string", "secure_loadfile: Argument 'file' is not a string.")
+
+    -- use the given (or create a new) restricted environment
+    local env = env or {}
+
+    -- load the file and run in a protected call with the restricted env
+    -- setfenv is deprecated in lua 5.2 in favor of giving env in arguments
+    -- the additional loadfile arguments are simply ignored for previous lua versions
+    local f, err = loadfile(file, 'bt', env)
+    if f then
+        if setfenv ~= nil then
+            setfenv(f, env)
+        end
+        return pcall(f)
+    else
+        return nil, err
+    end
+end
+
+-- Load and return manifest table from the manifest file.
+-- If manifest file not present, return nil.
+function load_manifest(manifest_file)
+    manifest_file = manifest_file or sys.make_path(cfg.root_dir, cfg.manifest_file)
+
+    return secure_loadfile(sys.abs_path(manifest_file))
+end
+
+-- Load '.gitmodules' file and returns manifest table.
+-- If the file is not present, return nil.
+function load_gitmodules(gitmodules_file)
+    gitmodules_file = gitmodules_file or sys.make_path(cfg.root_dir, cfg.manifest_file)
+    assert(type(gitmodules_file) == "string", "manifest.load_gitmodules: Argument 'gitmodules_file' is not a string.")
+    gitmodules_file = sys.abs_path(gitmodules_file)
+
+    if sys.exists(gitmodules_file) then
+        -- load the .gitmodules file
+        local file, err = io.open(gitmodules_file, "r")
+        if not file then return nil, "Error when opening the .gitmodules file '" .. gitmodules_file .. "':" .. err end
+
+        local mf_text = file:read("*a")
+        file:close()
+        if not mf_text then return nil, "Error when reading the .gitmodules file '" .. gitmodules_file .. "':" .. err end
+
+        manifest = {}
+        for url in mf_text:gmatch("git://%S+/%S+") do
+            pkg = {name = url:match("git://%S+/(%S+)%.git") or url:match("git://%S+/(%S+)"), version = "scm", path = url}
+            table.insert(manifest, pkg)
+        end
+
+        return manifest
+    else
+        return nil, "Error when loading the .gitmodules: file '" .. gitmodules_file .. "' doesn't exist."
+    end
+end
+
+-- Save manifest table to the 'file'
+function save_manifest(manifest_table, file)
+    assert(type(manifest_table) == "table", "manifest.save_distinfo: Argument 'manifest_table' is not a table.")
+    assert(type(file) == "string", "manifest.save_distinfo: Argument 'file' is not a string.")
+    file = sys.abs_path(file)
+
+    -- Print table 'tbl' to io stream 'file'.
+    local function print_table(file, tbl, in_nested_table)
+        for k, v in pairs(tbl) do
+            -- print key
+            if in_nested_table then file:write("\t\t") end
+            if type(k) ~= "number" then
+                file:write("['" .. k .. "']" .. " = ")
+            end
+            -- print value
+            if type(v) == "table" then
+                file:write("{\n")
+                print_table(file, v, true)
+                if in_nested_table then file:write("\t") end
+                file:write("\t}")
+            else
+                if in_nested_table then file:write("\t") end
+                if type(v) == "string" then
+                    file:write('[[' .. v .. ']]')
+                else
+                    file:write(tostring(v))
+                end
+            end
+            file:write(",\n")
+        end
+    end
+
+    local manifest_file = io.open(file, "w")
+    if not manifest_file then return nil, "Error when saving manifest: cannot open the file '" .. file .. "'." end
+
+    manifest_file:write('return {\n')
+    print_table(manifest_file, manifest_table)
+    manifest_file:write('},\ntrue')
+    manifest_file:close()
+
+    return true
+end
+
+-- Load and return package info table from the distinfo_file file.
+-- If file not present, return nil.
+function load_distinfo(distinfo_file)
+    assert(type(distinfo_file) == "string", "manifest.load_distinfo: Argument 'distinfo_file' is not a string.")
+    distinfo_file = sys.abs_path(distinfo_file)
+
+    -- load the distinfo file
+    local distinfo_env = {}
+    local status, ret = secure_loadfile(distinfo_file, distinfo_env)
+    if not status then return nil, "Error when loading package info: " .. ret end
+
+    return distinfo_env
+end
+
+-- Save distinfo table to the 'file'
+function save_distinfo(distinfo_table, file)
+    assert(type(distinfo_table) == "table", "manifest.save_distinfo: Argument 'distinfo_table' is not a table.")
+    assert(type(file) == "string", "manifest.save_distinfo: Argument 'file' is not a string.")
+    file = sys.abs_path(file)
+
+    -- Print table 'tbl' to io stream 'file'.
+    local function print_table(file, tbl, in_nested_table)
+        for k, v in pairs(tbl) do
+            -- print key
+            if type(k) ~= "number" then
+                file:write(k .. " = ")
+            end
+            -- print value
+            if type(v) == "table" then
+                file:write("{\n")
+                print_table(file, v, true)
+                file:write("}\n")
+            elseif type(v) == "string" then
+                if in_nested_table then
+                    file:write('[[' .. v .. ']]')
+                else
+                    file:write('"' .. v .. '"')
+                end
+            else
+                file:write(v)
+            end
+            if in_nested_table then
+                file:write(",")
+            end
+            file:write("\n")
+        end
+    end
+
+    local distinfo_file = io.open(file, "w")
+    if not distinfo_file then return nil, "Error when saving dist-info table: cannot open the file '" .. file .. "'." end
+
+    print_table(distinfo_file, distinfo_table)
+    distinfo_file:close()
+
+    return true
+end

lualibs/dist/package.lua

@@ -0,0 +1,596 @@
+-- Package functions
+
+module ("dist.package", package.seeall)
+
+local cfg = require "dist.config"
+local git = require "dist.git"
+local sys = require "dist.sys"
+local mf = require "dist.manifest"
+local utils = require "dist.utils"
+local depends = require "dist.depends"
+
+-- Return whether the package in given 'pkg_dir' is of a source type.
+function is_source_type(pkg_dir)
+    assert(type(pkg_dir) == "string", "package.is_source_type: Argument 'pkg_dir' is not a string.")
+    pkg_dir = sys.abs_path(pkg_dir)
+    return utils.to_boolean(sys.exists(sys.make_path(pkg_dir, "CMakeLists.txt")))
+end
+
+-- Ensure proper arch and type for the given source 'dist_info' table and return it.
+-- WARNING: this function should be used only for 'dist_info' tables of modules that are of a source type!
+function ensure_source_arch_and_type(dist_info)
+    assert(type(dist_info) == "table", "package.ensure_source_arch_and_type: Argument 'dist_info' is not a table.")
+    dist_info.arch = dist_info.arch or "Universal"
+    dist_info.type = dist_info.type or "source"
+    return dist_info
+end
+
+-- Remove package from 'pkg_distinfo_dir' of 'deploy_dir'.
+function remove_pkg(pkg_distinfo_dir, deploy_dir)
+    deploy_dir = deploy_dir or cfg.root_dir
+    assert(type(pkg_distinfo_dir) == "string", "package.remove_pkg: Argument 'pkg_distinfo_dir' is not a string.")
+    assert(type(deploy_dir) == "string", "package.remove_pkg: Argument 'deploy_dir' is not a string.")
+    deploy_dir = sys.abs_path(deploy_dir)
+
+    local abs_pkg_distinfo_dir = sys.make_path(deploy_dir, pkg_distinfo_dir)
+
+    -- check for 'dist.info'
+    local info, err = mf.load_distinfo(sys.make_path(abs_pkg_distinfo_dir, "dist.info"))
+    if not info then return nil, "Error removing package from '" .. pkg_distinfo_dir .. "' - it doesn't contain valid 'dist.info' file." end
+    if not info.files then return nil, "File '" .. sys.make_path(pkg_distinfo_dir, "dist.info") .."' doesn't contain list of installed files." end
+
+    -- remove files installed as components of this package
+    for _, component in ipairs(cfg.components) do
+        if info.files[component] then
+            for i = #info.files[component], 1, -1 do
+                local f = info.files[component][i]
+                f = sys.make_path(deploy_dir,f)
+                if sys.is_file(f) then
+                    sys.delete(f)
+                elseif sys.is_dir(f) then
+                    local dir_files, err = sys.get_file_list(f)
+                    if not dir_files then return nil, "Error removing package in '" .. abs_pkg_distinfo_dir .. "': " .. err end
+                    if #dir_files == 0 then sys.delete(f) end
+                end
+                -- delete also all parent directories if empty
+                local parents = sys.parents_up_to(f, deploy_dir)
+                for _, parent in ipairs(parents) do
+                    if sys.is_dir(parent) then
+                        local dir_files, err = sys.get_file_list(parent)
+                        if not dir_files then return nil, "Error removing package in '" .. abs_pkg_distinfo_dir .. "': " .. err end
+                        if #dir_files == 0 then
+                            sys.delete(parent)
+                        end
+                    end
+                end
+            end
+        end
+    end
+
+    -- remove removed components also from 'dist.info'
+    for _, component in ipairs(cfg.components) do
+        info.files[component] = nil
+    end
+
+    -- delete the package information from deploy_dir
+    local ok = sys.delete(abs_pkg_distinfo_dir)
+    if not ok then return nil, "Error removing package in '" .. abs_pkg_distinfo_dir .. "'." end
+
+    -- if the package was not completely removed (e.g. some components remain),
+    -- save the new version of its 'dist.info'
+    local comp_num = 0
+    for _, _ in pairs(info.files) do comp_num  = comp_num + 1 end
+    if comp_num ~= 0 then
+        sys.make_dir(abs_pkg_distinfo_dir)
+        local ok, err = mf.save_distinfo(info, sys.make_path(abs_pkg_distinfo_dir, "dist.info"))
+        if not ok then return nil, "Error resaving the 'dist.info': " .. err end
+    end
+
+    return ok
+end
+
+-- Install package from 'pkg_dir' to 'deploy_dir', using optional CMake 'variables'.
+-- Optional 'preserve_pkg_dir' argument specified whether to preserve the 'pkg_dir'.
+function install_pkg(pkg_dir, deploy_dir, variables, preserve_pkg_dir)
+    deploy_dir = deploy_dir or cfg.root_dir
+    variables = variables or {}
+    preserve_pkg_dir = preserve_pkg_dir or false
+
+    assert(type(pkg_dir) == "string", "package.install_pkg: Argument 'pkg_dir' is not a string.")
+    assert(type(deploy_dir) == "string", "package.install_pkg: Argument 'deploy_dir' is not a string.")
+    assert(type(variables) == "table", "package.install_pkg: Argument 'variables' is not a table.")
+    assert(type(preserve_pkg_dir) == "boolean", "package.install_pkg: Argument 'preserve_pkg_dir' is not a boolean.")
+
+    pkg_dir = sys.abs_path(pkg_dir)
+    deploy_dir = sys.abs_path(deploy_dir)
+
+    -- check for dist.info
+    local info, err = mf.load_distinfo(sys.make_path(pkg_dir, "dist.info"))
+    if not info then return nil, "Error installing: the directory '" .. pkg_dir .. "' doesn't exist or doesn't contain valid 'dist.info' file." end
+
+    -- check if the package is source
+    if is_source_type(pkg_dir) then info = ensure_source_arch_and_type(info) end
+
+    -- check package's architecture
+    if not (info.arch == "Universal" or info.arch == cfg.arch) then
+        return nil, "Error installing '" .. info.name .. "-" .. info.version .. "': architecture '" .. info.arch .. "' is not suitable for this machine."
+    end
+
+    -- check package's type
+    if not (info.type == "all" or info.type == "source" or info.type == cfg.type) then
+        return nil, "Error installing '" .. info.name .. "-" .. info.version .. "': architecture type '" .. info.type .. "' is not suitable for this machine."
+    end
+
+    local ok, err
+
+    -- if package is of binary type, just deploy it
+    if info.type ~= "source" then
+        ok, err = deploy_binary_pkg(pkg_dir, deploy_dir)
+
+    -- else build and then deploy
+    else
+
+        -- check if we have cmake
+        ok = utils.system_dependency_available("cmake", "cmake --version")
+        if not ok then return nil, "Error when installing: Command 'cmake' not available on the system." end
+
+        -- set cmake variables
+        local cmake_variables = {}
+
+        -- set variables from config file
+        for k, v in pairs(cfg.variables) do
+            cmake_variables[k] = v
+        end
+
+        -- set variables specified as argument
+        for k, v in pairs(variables) do
+            cmake_variables[k] = v
+        end
+
+        cmake_variables.CMAKE_INCLUDE_PATH = table.concat({cmake_variables.CMAKE_INCLUDE_PATH or "", sys.make_path(deploy_dir, "include")}, ";")
+        cmake_variables.CMAKE_LIBRARY_PATH = table.concat({cmake_variables.CMAKE_LIBRARY_PATH or "", sys.make_path(deploy_dir, "lib"), sys.make_path(deploy_dir, "bin")}, ";")
+        cmake_variables.CMAKE_PROGRAM_PATH = table.concat({cmake_variables.CMAKE_PROGRAM_PATH or "", sys.make_path(deploy_dir, "bin")}, ";")
+
+        -- build the package and deploy it
+        ok, err = build_pkg(pkg_dir, deploy_dir, cmake_variables)
+        if not ok then return nil, err end
+
+    end
+
+    -- delete directory of fetched package
+    if not (cfg.debug or preserve_pkg_dir) then sys.delete(pkg_dir) end
+
+    return ok, err
+end
+
+-- Build and deploy package from 'src_dir' to 'deploy_dir' using 'variables'.
+-- Return directory to which the package was built or nil on error.
+-- 'variables' is table of optional CMake variables.
+function build_pkg(src_dir, deploy_dir, variables)
+    deploy_dir = deploy_dir or cfg.root_dir
+    variables = variables or {}
+
+    assert(type(src_dir) == "string", "package.build_pkg: Argument 'src_dir' is not a string.")
+    assert(type(deploy_dir) == "string", "package.build_pkg: Argument 'deploy_dir' is not a string.")
+    assert(type(variables) == "table", "package.build_pkg: Argument 'variables' is not a table.")
+
+    src_dir = sys.abs_path(src_dir)
+    deploy_dir = sys.abs_path(deploy_dir)
+
+    -- check for dist.info
+    local info, err = mf.load_distinfo(sys.make_path(src_dir, "dist.info"))
+    if not info then return nil, "Error building package from '" .. src_dir .. "': it doesn't contain valid 'dist.info' file." end
+    local pkg_name = info.name .. "-" .. info.version
+
+    -- set machine information
+    info.arch = cfg.arch
+    info.type = cfg.type
+
+    -- create CMake build dir
+    local cmake_build_dir = sys.abs_path(sys.make_path(deploy_dir, cfg.temp_dir, pkg_name .. "-CMake-build"))
+    sys.make_dir(cmake_build_dir)
+
+    -- create cmake cache
+    variables["CMAKE_INSTALL_PREFIX"] = deploy_dir
+    local cache_file = io.open(sys.make_path(cmake_build_dir, "cache.cmake"), "w")
+    if not cache_file then return nil, "Error creating CMake cache file in '" .. cmake_build_dir .. "'" end
+
+    -- Fill in cache variables
+    for k,v in pairs(variables) do
+        cache_file:write("SET(" .. k .. " " .. sys.quote(v):gsub("\\+", "/") .. " CACHE STRING \"\" FORCE)\n")
+    end
+
+    -- If user cache file is provided then append it
+    if cfg.cache_file ~= "" then
+        local user_cache = io.open(sys.abs_path(cfg.cache_file), "r")
+        if user_cache then
+            cache_file:write(user_cache:read("*all").."\n")
+            user_cache:close()
+        end
+    end
+    cache_file:close()
+
+    src_dir = sys.abs_path(src_dir)
+    print("Building " .. sys.extract_name(src_dir) .. "...")
+
+    -- set cmake cache command
+    local cache_command = cfg.cache_command
+    if cfg.debug then cache_command = cache_command .. " " .. cfg.cache_debug_options end
+
+    -- set cmake build command
+    local build_command = cfg.build_command
+    if cfg.debug then build_command = build_command .. " " .. cfg.build_debug_options end
+
+    -- set the cmake cache
+    local ok = sys.exec("cd " .. sys.quote(cmake_build_dir) .. " && " .. cache_command .. " " .. sys.quote(src_dir))
+    if not ok then return nil, "Error preloading the CMake cache script '" .. sys.make_path(cmake_build_dir, "cache.cmake") .. "'" end
+
+    -- build with cmake
+    ok = sys.exec("cd " .. sys.quote(cmake_build_dir) .. " && " .. build_command)
+    if not ok then return nil, "Error building with CMake in directory '" .. cmake_build_dir .. "'" end
+
+    -- if this is only simulation, exit sucessfully, skipping the next actions
+    if cfg.simulate then
+        return true, "Simulated build and deployment of package '" .. pkg_name .. "' sucessfull."
+    end
+
+    -- table to collect files installed in the components
+    info.files = {}
+
+    -- install the components
+    for _, component in ipairs(cfg.components) do
+        local strip_option = ""
+        if not cfg.debug and component ~= "Library" then strip_option = cfg.strip_option end
+
+        local ok = sys.exec("cd " .. sys.quote(cmake_build_dir) .. " && " .. cfg.cmake .. " " .. strip_option .. " " ..cfg.install_component_command:gsub("#COMPONENT#", component))
+
+        if not ok then return nil, "Error when installing the component '" .. component .. "' with CMake in directory '" .. cmake_build_dir .. "'" end
+
+        local install_mf = sys.make_path(cmake_build_dir, "install_manifest_" .. component .. ".txt")
+        local mf, err
+        local component_files = {}
+
+        -- collect files installed in this component
+        if sys.exists(install_mf) then
+            mf, err = io.open(install_mf, "r")
+            if not mf then return nil, "Error when opening the CMake installation manifest '" .. install_mf .. "': " .. err end
+            for line in mf:lines() do
+                line = sys.check_separators(line)
+                local file = line:gsub(utils.escape_magic(deploy_dir .. sys.path_separator()), "")
+                table.insert(component_files, file)
+            end
+            mf:close()
+
+            -- add list of component files to the 'dist.info'
+            if #component_files > 0 then info.files[component] = component_files end
+        end
+    end
+--    if bookmark == 0 then return nil, "Package did not install any files!" end
+
+    -- test with ctest
+    if cfg.test then
+        print("Testing " .. sys.extract_name(src_dir) .. " ...")
+        ok = sys.exec("cd " .. sys.quote(deploy_dir) .. " && " .. cfg.test_command)
+        if not ok then return nil, "Error when testing the module '" .. pkg_name .. "' with CTest." end
+    end
+
+    -- save modified 'dist.info' file
+    local pkg_distinfo_dir = sys.make_path(deploy_dir, cfg.distinfos_dir, pkg_name)
+    sys.make_dir(pkg_distinfo_dir)
+    ok, err = mf.save_distinfo(info, sys.make_path(pkg_distinfo_dir, "dist.info"))
+    if not ok then return nil, err end
+
+    -- clean up
+    if not cfg.debug then sys.delete(cmake_build_dir) end
+
+    return true, "Package '" .. pkg_name .. "' successfully builded and deployed to '" .. deploy_dir .. "'."
+end
+
+-- Deploy binary package from 'pkg_dir' to 'deploy_dir' by copying.
+function deploy_binary_pkg(pkg_dir, deploy_dir)
+    deploy_dir = deploy_dir or cfg.root_dir
+
+    assert(type(pkg_dir) == "string", "package.deploy_binary_pkg: Argument 'pkg_dir' is not a string.")
+    assert(type(deploy_dir) == "string", "package.deploy_binary_pkg: Argument 'deploy_dir' is not a string.")
+
+    pkg_dir = sys.abs_path(pkg_dir)
+    deploy_dir = sys.abs_path(deploy_dir)
+
+    -- check for dist.info
+    local info, err = mf.load_distinfo(sys.make_path(pkg_dir, "dist.info"))
+    if not info then return nil, "Error deploying package from '" .. pkg_dir .. "': it doesn't contain valid 'dist.info' file." end
+    local pkg_name = info.name .. "-" .. info.version
+
+    -- if this is only simulation, exit sucessfully, skipping the next actions
+    if cfg.simulate then
+        return true, "Simulated deployment of package '" .. pkg_name .. "' sucessfull."
+    end
+
+    -- copy all components of the module to the deploy_dir
+    for _, component in ipairs(cfg.components) do
+        if info.files[component] then
+            for _, file in ipairs(info.files[component]) do
+                local dest_dir = sys.make_path(deploy_dir, sys.parent_dir(file))
+
+                local ok, err = sys.make_dir(dest_dir)
+                if not ok then return nil, "Error when deploying package '" .. pkg_name .. "': cannot create directory '" .. dest_dir .. "': " .. err end
+
+                ok, err = sys.copy(sys.make_path(pkg_dir, file), dest_dir)
+                if not ok then return nil, "Error when deploying package '" .. pkg_name .. "': cannot copy file '" .. file .. "' to the directory '" .. dest_dir .. "': " .. err end
+            end
+        end
+    end
+
+    -- copy dist.info to register the module as installed
+    local pkg_distinfo_dir = sys.make_path(deploy_dir, cfg.distinfos_dir, pkg_name)
+    sys.make_dir(pkg_distinfo_dir)
+    ok, err = mf.save_distinfo(info, sys.make_path(pkg_distinfo_dir, "dist.info"))
+    if not ok then return nil, err end
+
+    return true, "Package '" .. pkg_name .. "' successfully deployed to '" .. deploy_dir .. "'."
+end
+
+-- Fetch package (table 'pkg') to download_dir. Return the original 'pkg' table
+-- with 'pkg.download_dir' containing path to the directory of the
+-- downloaded package.
+--
+-- When optional 'suppress_printing' parameter is set to true, then messages
+-- for the user won't be printed during run of this function.
+--
+-- If the 'pkg' already contains the information about download directory (pkg.download_dir),
+-- we assume the package was already downloaded there and won't download it again.
+function fetch_pkg(pkg, download_dir, suppress_printing)
+    download_dir = download_dir or sys.current_dir()
+    suppress_printing = suppress_printing or false
+    assert(type(pkg) == "table", "package.fetch_pkg: Argument 'pkg' is not a table.")
+    assert(type(download_dir) == "string", "package.fetch_pkg: Argument 'download_dir' is not a string.")
+    assert(type(suppress_printing) == "boolean", "package.fetch_pkg: Argument 'suppress_printing' is not a boolean.")
+    assert(type(pkg.name) == "string", "package.fetch_pkg: Argument 'pkg.name' is not a string.")
+    assert(type(pkg.version) == "string", "package.fetch_pkg: Argument 'pkg.version' is not a string.")
+
+    -- if the package is already downloaded don't download it again
+    if pkg.download_dir then return pkg end
+
+    assert(type(pkg.path) == "string", "package.fetch_pkg: Argument 'pkg.path' is not a string.")
+    download_dir = sys.abs_path(download_dir)
+
+    local pkg_full_name = pkg.name .. "-" .. pkg.version
+    local repo_url = pkg.path
+    local clone_dir = sys.abs_path(sys.make_path(download_dir, pkg_full_name))
+    pkg.download_dir = clone_dir
+
+    -- check if download_dir already exists, assuming the package was already downloaded
+    if sys.exists(sys.make_path(clone_dir, "dist.info")) then
+        if cfg.cache and not utils.cache_timeout_expired(cfg.cache_timeout, clone_dir) then
+            if not suppress_printing then print("'" .. pkg_full_name .. "' already in cache, skipping downloading (use '-cache=false' to force download).") end
+            return pkg
+        else
+            sys.delete(sys.make_path(clone_dir))
+        end
+    end
+
+    local bin_tag = pkg.version .. "-" .. cfg.arch .. "-" .. cfg.type
+    local use_binary = false
+
+    if cfg.binary then
+        -- check if binary version of the module for this arch & type available
+        local avail_tags, err = git.get_remote_tags(repo_url)
+        if not avail_tags then return nil, err end
+
+        if utils.contains(avail_tags, bin_tag) then
+            use_binary = true
+        end
+    end
+
+    -- init the git repository
+    local ok, err = git.create_repo(clone_dir)
+    if not ok then return nil, err end
+
+    -- Fetch the desired ref (from the pkg's remote repo) and checkout into it.
+
+    if use_binary then
+
+        if not suppress_printing then print("Getting " .. pkg_full_name .. " (binary)...") end
+
+        -- We fetch the binary tag.
+        local sha
+        if ok then sha, err = git.fetch_tag(clone_dir, repo_url, bin_tag) end
+        if sha then ok, err = git.checkout_sha(sha, clone_dir) end
+
+    elseif cfg.source then
+
+        if not suppress_printing then print("Getting " .. pkg_full_name .. " (source)...") end
+
+        -- If we want the 'scm' version, we fetch the 'master' branch, otherwise
+        -- we fetch the tag, matching the desired package version.
+        if ok and pkg.version ~= "scm" then
+            local sha
+            sha, err = git.fetch_tag(clone_dir, repo_url, pkg.version)
+            if sha then ok, err = git.checkout_sha(sha, clone_dir) end
+        elseif ok then
+            local sha
+            sha, err = git.fetch_branch(clone_dir, repo_url, "master")
+            if sha then ok, err = git.checkout_sha(sha, clone_dir) end
+        end
+
+    else
+        ok = false
+        if cfg.binary then
+            err = "Binary version of module not available and using source modules disabled."
+        else
+            err = "Using both binary and source modules disabled."
+        end
+    end
+
+    if not ok then
+        -- clean up
+        if not cfg.debug then sys.delete(clone_dir) end
+        return nil, "Error fetching package '" .. pkg_full_name .. "' from '" .. pkg.path .. "' to '" .. download_dir .. "': " .. err
+    end
+
+    -- delete '.git' directory
+    if not cfg.debug then sys.delete(sys.make_path(clone_dir, ".git")) end
+
+    return pkg
+end
+
+-- Return table with information about available versions of 'package'.
+--
+-- When optional 'suppress_printing' parameter is set to true, then messages
+-- for the user won't be printed during run of this function.
+function retrieve_versions(package, manifest, suppress_printing)
+    suppress_printing = suppress_printing or false
+    assert(type(package) == "string", "package.retrieve_versions: Argument 'string' is not a string.")
+    assert(type(manifest) == "table", "package.retrieve_versions: Argument 'manifest' is not a table.")
+    assert(type(suppress_printing) == "boolean", "package.retrieve_versions: Argument 'suppress_printing' is not a boolean.")
+
+    -- get package table
+    local pkg_name = depends.split_name_constraint(package)
+    local tmp_packages = depends.find_packages(pkg_name, manifest)
+
+    if #tmp_packages == 0 then
+        return nil, "No suitable candidate for package '" .. package .. "' found."
+    else
+        package = tmp_packages[1]
+    end
+
+    -- if the package's already downloaded, we assume it's desired to install the downloaded version
+    if package.download_dir then
+        local pkg_type = "binary"
+        if is_source_type(package.download_dir) then pkg_type = "source" end
+        if not suppress_printing then print("Using " .. package.name .. "-" .. package.version .. " (" .. pkg_type .. ") provided by " .. package.download_dir) end
+        return {package}
+    end
+
+    if not suppress_printing then print("Finding out available versions of " .. package.name .. "...") end
+
+    -- get available versions
+    local tags, err = git.get_remote_tags(package.path)
+    if not tags then return nil, "Error when retrieving versions of package '" .. package.name .. "': " .. err end
+
+    -- filter out tags of binary packages
+    local versions = utils.filter(tags, function (tag) return tag:match("^[^%-]+%-?[^%-]*$") and true end)
+
+    packages = {}
+
+    -- create package information
+    for _, version in pairs(versions) do
+        pkg = {}
+        pkg.name = package.name
+        pkg.version = version
+        pkg.path = package.path
+        table.insert(packages, pkg)
+    end
+
+    return packages
+end
+
+-- Return table with information from package's dist.info and path to downloaded
+-- package. Optional argument 'deploy_dir' is used just as a temporary
+-- place to place the downloaded packages into.
+--
+-- When optional 'suppress_printing' parameter is set to true, then messages
+-- for the user won't be printed during the execution of this function.
+function retrieve_pkg_info(package, deploy_dir, suppress_printing)
+    deploy_dir = deploy_dir or cfg.root_dir
+    assert(type(package) == "table", "package.retrieve_pkg_info: Argument 'package' is not a table.")
+    assert(type(deploy_dir) == "string", "package.retrieve_pkg_info: Argument 'deploy_dir' is not a string.")
+    deploy_dir = sys.abs_path(deploy_dir)
+
+    local tmp_dir = sys.abs_path(sys.make_path(deploy_dir, cfg.temp_dir))
+
+    -- download the package
+    local fetched_pkg, err = fetch_pkg(package, tmp_dir, suppress_printing)
+    if not fetched_pkg then return nil, "Error when retrieving the info about '" .. package.name .. "': " .. err end
+
+    -- load information from 'dist.info'
+    local info, err = mf.load_distinfo(sys.make_path(fetched_pkg.download_dir, "dist.info"))
+    if not info then return nil, err end
+
+    -- add other attributes
+    if package.path then info.path = package.path end
+    if package.was_scm_version then info.was_scm_version = package.was_scm_version end
+
+    -- set default arch/type if not explicitly stated and package is of source type
+    if is_source_type(fetched_pkg.download_dir) then
+        info = ensure_source_arch_and_type(info)
+    elseif not (info.arch and info.type) then
+        return nil, fetched_pkg.download_dir .. ": binary package missing arch or type in 'dist.info'."
+    end
+
+    return info, fetched_pkg.download_dir
+end
+
+-- Return manifest, augmented with info about all available versions
+-- of package 'pkg'. Optional argument 'deploy_dir' is used just as a temporary
+-- place to place the downloaded packages into.
+-- Optional argument 'installed' is manifest of all installed packages. When
+-- specified, info from installed packages won't be downloaded from repo,
+-- but the dist.info from installed package will be used.
+function get_versions_info(pkg, manifest, deploy_dir, installed)
+    deploy_dir = deploy_dir or cfg.root_dir
+    assert(type(pkg) == "string", "package.get_versions_info: Argument 'pkg' is not a string.")
+    assert(type(manifest) == "table", "package.get_versions_info: Argument 'manifest' is not a table.")
+    assert(type(deploy_dir) == "string", "package.get_versions_info: Argument 'deploy_dir' is not a string.")
+    deploy_dir = sys.abs_path(deploy_dir)
+
+    -- find all available versions of package
+    local versions, err = retrieve_versions(pkg, manifest)
+    if not versions then return nil, err end
+
+    -- collect info about all retrieved versions
+    local infos = {}
+    for _, version in pairs(versions) do
+
+        local info, path_or_err
+        local installed_version = {}
+
+        -- find out whether this 'version' is installed so we can use it's dist.info
+        if type(installed) == "table" then installed_version = depends.find_packages(version.name .. "-" .. version.version, installed) end
+
+        -- get info
+        if #installed_version > 0 then
+            print("Using dist.info from installed " .. version.name .. "-" .. version.version)
+            info = installed_version[1]
+            info.path = version.path
+            info.from_installed = true  -- flag that dist.info of installed package was used
+        else
+            info, path_or_err = retrieve_pkg_info(version, deploy_dir)
+            if not info then return nil, path_or_err end
+            sys.delete(path_or_err)
+        end
+        table.insert(infos, info)
+    end
+
+    -- found and add an implicit 'scm' version
+    local pkg_name = depends.split_name_constraint(pkg)
+    local found = depends.find_packages(pkg_name, manifest)
+    if #found == 0 then return nil, "No suitable candidate for package '" .. pkg .. "' found." end
+    local scm_info, path_or_err = retrieve_pkg_info({name = pkg_name, version = "scm", path = found[1].path})
+    if not scm_info then return nil, path_or_err end
+    sys.delete(path_or_err)
+    scm_info.version = "scm"
+    table.insert(infos, scm_info)
+
+    local tmp_manifest = utils.deepcopy(manifest)
+
+    -- add collected info to the temp. manifest, replacing existing tables
+    for _, info in pairs(infos) do
+        local already_in_manifest = false
+        -- find if this version is already in manifest
+        for idx, pkg in ipairs(tmp_manifest) do
+            -- if yes, replace it
+            if pkg.name == info.name and pkg.version == info.version then
+                tmp_manifest[idx] = info
+                already_in_manifest = true
+                break
+            end
+        end
+        -- if not, just normally add to the manifest
+        if not already_in_manifest then
+            table.insert(tmp_manifest, info)
+        end
+    end
+
+    return tmp_manifest
+end

lualibs/dist/sys.lua

@@ -0,0 +1,386 @@
+-- System functions
+
+module ("dist.sys", package.seeall)
+
+local cfg = require "dist.config"
+local utils = require "dist.utils"
+local lfs = require "lfs"
+
+-- Return the path separator according to the platform.
+function path_separator()
+    if cfg.arch == "Windows" then
+        return "\\"
+    else
+        return "/"
+    end
+end
+
+-- Return path with wrong separators replaced with the right ones.
+function check_separators(path)
+    assert(type(path) == "string", "sys.check_separators: Argument 'path' is not a string.")
+    if cfg.arch == "Windows" then
+        return path:gsub("/", "\\")
+    else
+        return path
+    end
+end
+
+-- Return the path with the unnecessary trailing separator removed.
+function remove_trailing(path)
+    assert(type(path) == "string", "sys.remove_trailing: Argument 'path' is not a string.")
+    if path:sub(-1) == path_separator() and not is_root(path) then path = path:sub(1,-2) end
+    return path
+end
+
+-- Return the path with the all occurences of '/.' or '\.' (representing
+-- the current directory) removed.
+function remove_curr_dir_dots(path)
+    assert(type(path) == "string", "sys.remove_curr_dir_dots: Argument 'path' is not a string.")
+    while path:match(path_separator() .. "%." .. path_separator()) do                       -- match("/%./")
+        path = path:gsub(path_separator() .. "%." .. path_separator(), path_separator())    -- gsub("/%./", "/")
+    end
+    return path:gsub(path_separator() .. "%.$", "")                                         -- gsub("/%.$", "")
+end
+
+-- Return string argument quoted for a command line usage.
+function quote(argument)
+    assert(type(argument) == "string", "sys.quote: Argument 'argument' is not a string.")
+
+    -- TODO: This seems like a not very nice hack. Why is it needed?
+    --       Wouldn't it be better to fix the problem where it originates?
+    -- replace '/' path separators for '\' on Windows
+    if cfg.arch == "Windows" and argument:match("^[%u%U.]?:?[/\\].*") then
+        argument = argument:gsub("//","\\"):gsub("/","\\")
+    end
+
+    -- Windows doesn't recognize paths starting with two slashes or backslashes
+    -- so we double every backslash except for the first one
+    if cfg.arch == "Windows" and argument:match("^[/\\].*") then
+        local prefix = argument:sub(1,1)
+        argument = argument:sub(2):gsub("\\",  "\\\\")
+        argument = prefix .. argument
+    else
+        argument = argument:gsub("\\",  "\\\\")
+    end
+    argument = argument:gsub('"',  '\\"')
+
+    return '"' .. argument .. '"'
+end
+
+-- Run the system command (in current directory).
+-- Return true on success, nil on fail and log string.
+-- When optional 'force_verbose' parameter is true, then the output will be shown
+-- even when not in debug or verbose mode.
+function exec(command, force_verbose)
+    force_verbose = force_verbose or false
+    assert(type(command) == "string", "sys.exec: Argument 'command' is not a string.")
+    assert(type(force_verbose) == "boolean", "sys.exec: Argument 'force_verbose' is not a boolean.")
+
+    if not (cfg.verbose or cfg.debug or force_verbose) then
+        if cfg.arch == "Windows" then
+            command = command .. " > NUL 2>&1"
+        else
+            command = command .. " > /dev/null 2>&1"
+        end
+    end
+
+    if cfg.debug then print("Executing the command: " .. command) end
+    local ok, str, status  = os.execute(command)
+
+    -- os.execute returned values on failure are:
+    --  nil or true, "exit", n or true, "signal", n for lua >= 5.2
+    --  status ~= 0 for lua 5.x < 5.2
+    if ok == nil or (str == "exit" and status ~= 0) or str == "signal" or (ok ~= 0 and ok ~= true) then
+        return nil, "Error when running the command: " .. command
+    else
+        return true, "Sucessfully executed the command: " .. command
+    end
+end
+
+-- Execute the 'command' and returns its output as a string.
+function capture_output(command)
+    assert(type(command) == "string", "sys.exec: Argument 'command' is not a string.")
+
+    local executed, err = io.popen(command, "r")
+    if not executed then return nil, "Error running the command '" .. command .. "':" .. err end
+
+    local captured, err = executed:read("*a")
+    if not captured then return nil, "Error reading the output of command '" .. command .. "':" .. err end
+
+    executed:close()
+    return captured
+end
+
+-- Return whether the path is a root.
+function is_root(path)
+    assert(type(path) == "string", "sys.is_root: Argument 'path' is not a string.")
+    return utils.to_boolean(path:find("^[a-zA-Z]:[/\\]$") or path:find("^[/\\]$"))
+end
+
+-- Return whether the path is absolute.
+function is_abs(path)
+    assert(type(path) == "string", "sys.is_abs: Argument 'path' is not a string.")
+    return utils.to_boolean(path:find("^[a-zA-Z]:[/\\].*$") or path:find("^[/\\].*$"))
+end
+
+-- Return whether the specified file or directory exists.
+function exists(path)
+    assert(type(path) == "string", "sys.exists: Argument 'path' is not a string.")
+    local attr, err = lfs.attributes(path)
+    return utils.to_boolean(attr), err
+end
+
+-- Return whether the 'file' exists and is a file.
+function is_file(file)
+    assert(type(file) == "string", "sys.is_file: Argument 'file' is not a string.")
+    return lfs.attributes(file, "mode") == "file"
+end
+
+-- Return whether the 'dir' exists and is a directory.
+function is_dir(dir)
+    assert(type(dir) == "string", "sys.is_dir: Argument 'dir' is not a string.")
+    return lfs.attributes(dir, "mode") == "directory"
+end
+
+-- Return the current working directory
+function current_dir()
+    local dir, err = lfs.currentdir()
+    if not dir then return nil, err end
+    return dir
+end
+
+-- Return an iterator over the directory 'dir'.
+-- If 'dir' doesn't exist or is not a directory, return nil and error message.
+function get_directory(dir)
+    dir = dir or current_dir()
+    assert(type(dir) == "string", "sys.get_directory: Argument 'dir' is not a string.")
+    if is_dir(dir) then
+        return lfs.dir(dir)
+    else
+        return nil, "Error: '".. dir .. "' is not a directory."
+    end
+end
+
+-- Extract file or directory name from its path.
+function extract_name(path)
+    assert(type(path) == "string", "sys.extract_name: Argument 'path' is not a string.")
+    if is_root(path) then return path end
+
+    path = remove_trailing(path)
+    path = path:gsub("^.*" .. path_separator(), "")
+    return path
+end
+
+-- Return parent directory of the 'path' or nil if there's no parent directory.
+-- If 'path' is a path to file, return the directory the file is in.
+function parent_dir(path)
+    assert(type(path) == "string", "sys.parent_dir: Argument 'path' is not a string.")
+    path = remove_curr_dir_dots(path)
+    path = remove_trailing(path)
+
+    local dir = path:gsub(utils.escape_magic(extract_name(path)) .. "$", "")
+    if dir == "" then
+        return nil
+    else
+        return make_path(dir)
+    end
+end
+
+-- Returns the table of all parent directories of 'path' up to the directory
+-- specified by 'boundary_path' (exclusive).
+function parents_up_to(path, boundary_path)
+    assert(type(path) == "string", "sys.parents_up_to: Argument 'path' is not a string.")
+    assert(type(boundary_path) == "string", "sys.parents_up_to: Argument 'boundary_path' is not a string.")
+    boundary_path = remove_trailing(boundary_path)
+
+    -- helper function to recursively collect the parent directories
+    local function collect_parents(_path, _parents)
+        local _parent = parent_dir(_path)
+        if _parent and _parent ~= boundary_path then
+            table.insert(_parents, _parent)
+            return collect_parents(_parent, _parents)
+        else
+            return _parents
+        end
+    end
+
+    return collect_parents(path, {})
+end
+
+-- Compose path composed from specified parts or current
+-- working directory when no part specified.
+function make_path(...)
+    -- arg is deprecated in lua 5.2 in favor of table.pack we mimic here
+    local arg = {n=select('#',...),...}
+    local parts = arg
+    assert(type(parts) == "table", "sys.make_path: Argument 'parts' is not a table.")
+
+    local path, err
+    if parts.n == 0 then
+        path, err = current_dir()
+    else
+        path, err = table.concat(parts, path_separator())
+    end
+    if not path then return nil, err end
+
+    -- squeeze repeated occurences of a file separator
+    path = path:gsub(path_separator() .. "+", path_separator())
+
+    -- remove unnecessary trailing path separator
+    path = remove_trailing(path)
+
+    return path
+end
+
+-- Return absolute path from 'path'
+function abs_path(path)
+    assert(type(path) == "string", "sys.get_abs_path: Argument 'path' is not a string.")
+    if is_abs(path) then return path end
+
+    local cur_dir, err = current_dir()
+    if not cur_dir then return nil, err end
+
+    return make_path(cur_dir, path)
+end
+
+-- Returns path to the temporary directory of OS.
+function tmp_dir()
+    return os.getenv("TMPDIR") or os.getenv("TEMP") or os.getenv("TMP") or "/tmp"
+end
+
+-- Returns temporary file (or directory) path (with optional prefix).
+function tmp_name(prefix)
+    prefix = prefix or ""
+    assert(type(prefix) == "string", "sys.tmp_name: Argument 'prefix' is not a string.")
+    return make_path(tmp_dir(), prefix .. "luadist_" .. utils.rand(10000000000))
+end
+
+-- Return table of all paths in 'dir'
+function get_file_list(dir)
+    dir = dir or current_dir()
+    assert(type(dir) == "string", "sys.get_directory: Argument 'dir' is not a string.")
+    if not exists(dir) then return nil, "Error getting file list of '" .. dir .. "': directory doesn't exist." end
+
+    local function collect(path, all_paths)
+        for item in get_directory(path) do
+
+            local item_path = make_path(path, item)
+            local _, last = item_path:find(dir .. path_separator(), 1, true)
+            local path_to_insert = item_path:sub(last + 1)
+
+            if is_file(item_path) then
+                table.insert(all_paths, path_to_insert)
+            elseif is_dir(item_path) and item ~= "." and item ~= ".." then
+                table.insert(all_paths, path_to_insert)
+                collect(item_path, all_paths)
+            end
+        end
+    end
+
+    local all_paths = {}
+    collect(dir, all_paths)
+
+    return all_paths
+end
+
+-- Return time of the last modification of 'file'.
+function last_modification_time(file)
+    assert(type(file) == "string", "sys.last_modification_time: Argument 'file' is not a string.")
+    return lfs.attributes(file, "modification")
+end
+
+-- Return the current time (in seconds since epoch).
+function current_time()
+    return os.time()
+end
+
+-- Change the current working directory and return 'true' and previous working
+-- directory on success and 'nil' and error message on error.
+function change_dir(dir_name)
+    assert(type(dir_name) == "string", "sys.change_dir: Argument 'dir_name' is not a string.")
+    local prev_dir = current_dir()
+    local ok, err = lfs.chdir(dir_name)
+    if ok then
+        return ok, prev_dir
+    else
+        return nil, err
+    end
+end
+
+-- Make a new directory, making also all of its parent directories that doesn't exist.
+function make_dir(dir_name)
+    assert(type(dir_name) == "string", "sys.make_dir: Argument 'dir_name' is not a string.")
+    if exists(dir_name) then
+        return true
+    else
+        local par_dir = parent_dir(dir_name)
+        if par_dir then
+            local ok, err = make_dir(par_dir)
+            if not ok then return nil, err end
+        end
+        return lfs.mkdir(dir_name)
+    end
+end
+
+-- Move file (or directory) to the destination directory
+function move_to(file_or_dir, dest_dir)
+    assert(type(file_or_dir) == "string", "sys.move_to: Argument 'file_or_dir' is not a string.")
+    assert(type(dest_dir) == "string", "sys.move_to: Argument 'dest_dir' is not a string.")
+    assert(is_dir(dest_dir), "sys.move_to: Destination '" .. dest_dir .."' is not a directory.")
+
+    -- Extract file/dir name from its path
+    local file_or_dir_name = extract_name(file_or_dir)
+
+    return os.rename(file_or_dir, make_path(dest_dir, file_or_dir_name))
+end
+
+-- rename file (or directory) to the new name.
+function rename(file, new_name)
+    assert(type(file) == "string", "sys.rename: Argument 'file' is not a string.")
+    assert(type(new_name) == "string", "sys.rename: Argument 'new_name' is not a string.")
+    assert(not exists(new_name), "sys.rename: desired filename already exists.")
+
+    return os.rename(file, new_name)
+end
+
+-- Copy 'source' to the destination directory 'dest_dir'.
+-- If 'source' is a directory, then recursive copying is used.
+-- For non-recursive copying of directories use the make_dir() function.
+function copy(source, dest_dir)
+    assert(type(source) == "string", "sys.copy: Argument 'file_or_dir' is not a string.")
+    assert(type(dest_dir) == "string", "sys.copy: Argument 'dest_dir' is not a string.")
+    assert(is_dir(dest_dir), "sys.copy: destination '" .. dest_dir .."' is not a directory.")
+
+    if cfg.arch == "Windows" then
+        if is_dir(source) then
+            make_dir(make_path(dest_dir, extract_name(source)))
+            return exec("xcopy /E /I /Y /Q " .. quote(source) .. " " .. quote(dest_dir .. "\\" .. extract_name(source)))
+        else
+            return exec("copy /Y " .. quote(source) .. " " .. quote(dest_dir))
+        end
+    else
+        if is_dir(source) then
+            return exec("cp -fRH " .. quote(source) .. " " .. quote(dest_dir))
+        else
+            return exec("cp -fH " .. quote(source) .. " " .. quote(dest_dir))
+        end
+    end
+end
+
+-- Delete the specified file or directory
+function delete(path)
+    assert(type(path) == "string", "sys.delete: Argument 'path' is not a string.")
+    assert(is_abs(path), "sys.delete: Argument 'path' is not an absolute path.")
+
+    if cfg.arch == "Windows" then
+        if not exists(path) then
+            return true
+        elseif is_file(path) then
+            return os.remove(path)
+        else
+            return exec("rd /S /Q " .. quote(path))
+        end
+    else
+        return exec("rm -rf " .. quote(path))
+    end
+end

lualibs/dist/utils.lua

@@ -0,0 +1,151 @@
+-- System functions
+
+module ("dist.utils", package.seeall)
+
+local sys = require "dist.sys"
+
+-- Returns a deep copy of 'table' with reference to the same metadata table.
+-- Source: http://lua-users.org/wiki/CopyTable
+function deepcopy(object)
+    local lookup_table = {}
+    local function _copy(object)
+        if type(object) ~= "table" then
+            return object
+        elseif lookup_table[object] then
+            return lookup_table[object]
+        end
+        local new_table = {}
+        lookup_table[object] = new_table
+        for index, value in pairs(object) do
+            new_table[_copy(index)] = _copy(value)
+        end
+        return setmetatable(new_table, getmetatable(object))
+    end
+    return _copy(object)
+end
+
+-- Return deep copy of table 'array', containing only items for which 'predicate_fn' returns true.
+function filter(array, predicate_fn)
+    assert(type(array) == "table", "utils.filter: Argument 'array' is not a table.")
+    assert(type(predicate_fn) == "function", "utils.filter: Argument 'predicate_fn' is not a function.")
+    local filtered = {}
+    for _,v in pairs(array) do
+        if predicate_fn(v) == true then table.insert(filtered, deepcopy(v)) end
+    end
+    return filtered
+end
+
+-- Return deep copy of table 'array', sorted according to the 'compare_fn' function.
+function sort(array, compare_fn)
+    assert(type(array) == "table", "utils.sort: Argument 'array' is not a table.")
+    assert(type(compare_fn) == "function", "utils.sort: Argument 'compare_fn' is not a function.")
+    local sorted = deepcopy(array)
+    table.sort(sorted, compare_fn)
+    return sorted
+end
+
+-- Return whether the 'value' is in the table 'tbl'.
+function contains(tbl, value)
+    assert(type(tbl) == "table", "utils.contains: Argument 'tbl' is not a table.")
+    for _,v in pairs(tbl) do
+        if v == value then return true end
+    end
+    return false
+end
+
+-- Return single line string consisting of values in 'tbl' separated by comma.
+-- Used for printing the dependencies/provides/conflicts.
+function table_tostring(tbl, label)
+    assert(type(tbl) == "table", "utils.table_tostring: Argument 'tbl' is not a table.")
+    local str = ""
+    for k,v in pairs(tbl) do
+        if type(v) == "table" then
+            str = str .. table_tostring(v, k)
+        else
+            if label ~= nil then
+                str = str .. tostring(v) .. " [" .. tostring(label) .. "]" .. ", "
+            else
+                str = str .. tostring(v) .. ", "
+            end
+        end
+    end
+    return str
+end
+
+-- Return table made up from values of the string, separated by separator.
+function make_table(str, separator)
+    assert(type(str) == "string", "utils.make_table: Argument 'str' is not a string.")
+    assert(type(separator) == "string", "utils.make_table: Argument 'separator' is not a string.")
+
+    local tbl = {}
+    for val in str:gmatch("(.-)" .. separator) do
+        table.insert(tbl, val)
+    end
+    local last_val = str:gsub(".-" .. separator, "")
+    if last_val and last_val ~= "" then
+        table.insert(tbl, last_val)
+    end
+    return tbl
+end
+
+-- Return whether the 'cache_timeout' for 'file' has expired.
+function cache_timeout_expired(cache_timeout, file)
+    assert(type(cache_timeout) == "number", "utils.cache_timeout_expired: Argument 'cache_timeout' is not a number.")
+    assert(type(file) == "string", "utils.cache_timeout_expired: Argument 'file' is not a string.")
+    return sys.last_modification_time(file) + cache_timeout < sys.current_time()
+end
+
+-- Return the string 'str', with all magic (pattern) characters escaped.
+function escape_magic(str)
+    assert(type(str) == "string", "utils.escape: Argument 'str' is not a string.")
+    local escaped = str:gsub('[%-%.%+%[%]%(%)%^%%%?%*%^%$]','%%%1')
+    return escaped
+end
+
+-- Return the boolean representation of an 'arg'.
+function to_boolean(arg)
+    return not not arg
+end
+
+
+math.randomseed(os.time())
+
+-- Return pseudo-random number in range [0, 1], [1, n] or [n, m].
+function rand(...)
+    return math.random(...)
+end
+
+-- Perform check of system dependency, which isn't provided in the LuaDist
+-- installation itself and if it is missing, print instructions how
+-- to install it. The 'command' is used for testing, 'name' when printing
+-- information to the user.
+function system_dependency_available(name, command)
+    assert(type(name) == "string", "utils.system_dependency_available: Argument 'name' is not a string.")
+    assert(type(command) == "string", "utils.system_dependency_available: Argument 'command' is not a string.")
+
+    if not sys.exec(command) then
+        print("Error: command '" .. name .. "' not found on system. See installation instructions at\nhttps://github.com/LuaDist/Repository/wiki/Installation-of-System-Dependencies")
+        return false
+    end
+
+    return true
+end
+
+-- Obtain LuaDist location by checking available package locations
+function get_luadist_location()
+  local paths = {}
+  local path = package.path:gsub("([^;]+)", function(c) table.insert(paths, c) end)
+
+  for _, path in pairs(paths) do
+    if (sys.is_abs(path) and path:find("[/\\]lib[/\\]lua[/\\]%?.lua$")) then
+      -- Remove path to lib/lua
+      path = path:gsub("[/\\]lib[/\\]lua[/\\]%?.lua$", "")
+      -- Clean the path up a bit
+      path = path:gsub("[/\\]bin[/\\]%.[/\\]%.%.", "")
+      path = path:gsub("[/\\]bin[/\\]%.%.", "")
+      return path
+    end
+  end
+  return nil
+end
+

lualibs/git.lua

@@ -0,0 +1,5 @@
+require 'git.util'
+require 'git.objects'
+require 'git.pack'
+require 'git.repo'
+require 'git.protocol'

lualibs/git/objects.lua

@@ -0,0 +1,121 @@
+local util = require 'git.util'
+
+local assert, next, io, print, os, type, string, pairs, tostring =
+	assert, next, io, print, os, type, string, pairs, tostring
+local join_path = git.util.join_path
+
+local require = require
+
+local isPosix = package.config:sub(1,1) == '/' -- wild guess
+
+module(...)
+
+Commit = {}
+Commit.__index = Commit
+
+function Commit:tree()
+	return self.repo:tree(self.tree_sha)
+end
+
+function Commit:checkout(path)
+	assert(path, 'path argument missing')
+	self:tree():checkoutTo(path)
+end
+
+
+Tree = {}
+Tree.__index = function (t,k)
+	if Tree[k] then return Tree[k] end
+	return t:entry(k)
+end
+
+function Tree:entries()
+	return function(t, n)
+		local n, entry = next(t, n)
+		if entry then
+			local object
+			if entry.type == 'tree' then
+				object = self.repo:tree(entry.id)
+			elseif entry.type == 'blob' then
+				object = self.repo:blob(entry.id)
+				object.mode = entry.mode
+			elseif entry.type == 'commit' then
+				-- this is possibly a commit in a submodule, 
+				-- do not retrieve it from current repo
+				object = entry
+			else
+				error('Unknown entry type: ' .. entry.type)
+			end
+			return n, entry.type, object
+		end
+	end, self._entries
+end
+
+function Tree:entry(n)
+	local e = self._entries[n]
+	if not e then return end
+	if e.type == 'tree' then
+		return self.repo:tree(e.id)
+	elseif e.type == 'commit' then
+		return self.repo:commit(e.id)
+	elseif e.type == 'blob' then
+		return self.repo:blob(e.id)
+	else
+		error('Unknown entry type: ' .. e.type)
+	end
+end
+
+function Tree:walk(func, path)
+	path = path or '.'
+	assert(type(func) == "function", "argument is not a function")
+	local function walk(tree, path)
+		for name, type, entry in tree:entries() do
+			local entry_path = join_path(path, name)
+			func(entry, entry_path, type)
+
+			if type == "tree" then
+				walk(entry, entry_path)
+			end
+		end
+	end
+	walk(self, path)
+end
+
+function Tree:checkoutTo(path)
+	util.make_dir(path)
+	self:walk(function (entry, entry_path, type)
+		if type == 'tree' then
+			util.make_dir(entry_path)
+		elseif type == 'blob' then
+			local out = assert(io.open(entry_path, 'wb'))
+			out:write(entry:content())
+			out:close()
+			if isPosix then
+				local mode = entry.mode:sub(-3,-1) -- fixme: is this ok?
+				local cmd = 'chmod '..mode..' "'..entry_path..'"'
+				os.execute(cmd)
+			end
+		elseif type == 'commit' then
+			-- this is a submodule referencing a commit,
+			-- make a directory for it
+			util.make_dir(entry_path)
+		else
+			error('Unknown entry type: ', type)
+		end
+	end, path)
+end
+
+Blob = {}
+Blob.__index = Blob
+
+function Blob:content()
+	if self.stored then
+		local f = self.repo:raw_object(self.id)
+		local ret = f:read('*a') or ""
+		f:close()
+		return ret
+	else
+		return self.data
+	end
+end
+

lualibs/git/pack.lua

@@ -0,0 +1,316 @@
+local io = io
+local core = require 'git.core'
+
+local assert, pcall, print, select, setmetatable, string, type, unpack =
+	assert, pcall, print, select, setmetatable, string, type, unpack
+
+local ord = string.byte
+local fmt = string.format
+local concat, insert = table.concat, table.insert
+
+local band = core.band
+local rshift, lshift = core.rshift, core.lshift
+
+local to_hex = git.util.to_hex
+local from_hex = git.util.from_hex
+local object_sha = git.util.object_sha
+local binary_sha = git.util.binary_sha
+local readable_sha = git.util.readable_sha
+local tmpfile = git.util.tmpfile
+local reader = git.util.reader
+
+module(...)
+
+-- read git/Documentation/technical/pack-format.txt for some inspiration
+
+-- 1 = commit, 2 = tree ...
+local types = {'commit', 'tree', 'blob', 'tag', '???', 'ofs_delta', 'ref_delta'}
+
+-- read a 4 byte unsigned integer stored in network order
+local function read_int(f)
+	local s = f:read(4)
+	local a,b,c,d = s:byte(1,4)
+	return a*256^3 + b*256^2 + c*256 + d
+end
+
+-- read in the type and file length
+local function read_object_header(f)
+	local b = ord(f:read(1))
+	local type = band(rshift(b, 4), 0x7)
+	local len = band(b, 0xF)
+	local ofs = 0
+	while band(b, 0x80) ~= 0 do
+		b = ord(f:read(1))
+		len = len + lshift(band(b, 0x7F), ofs * 7 + 4)
+		ofs = ofs + 1
+	end
+	return len, type
+end
+
+-- reads in the delta header and returns the offset where original data is stored
+local function read_delta_header(f)
+	local b = ord(f:read(1))
+	local offset = band(b, 0x7F)
+	while band(b, 0x80) ~= 0 do
+		offset = offset + 1
+		b = ord(f:read(1))
+		offset = lshift(offset, 7) + band(b, 0x7F)
+	end
+	return offset
+end
+
+-- read just enough of file `f` to uncompress `size` bytes
+local function uncompress_by_len(f, size)
+	local z = core.inflate()
+	local chunks = {}
+	local CHUNK_SIZE = 1024
+	local curr_pos = f:seek()
+	local inflated, eof, total
+	-- read until end of zlib-compresed stream
+	while not eof do
+		local data = f:read(CHUNK_SIZE)
+		inflated, eof, total = z(data)
+		insert(chunks, inflated)
+	end
+	-- repair the current position in stream
+	f:seek('set', curr_pos + total)
+	return concat(chunks)
+end
+
+-- uncompress the object from the current location in `f`
+local function unpack_object(f, len, type)
+	local data = uncompress_by_len(f, len)
+	return data, len, type
+end
+
+-- returns a size value encoded in delta data
+local function delta_size(f)
+	local size = 0
+	local i = 0
+	repeat
+		local b = ord(f:read(1))
+		size = size + lshift(band(b, 0x7F), i)
+		i = i + 7
+	until band(b, 0x80) == 0
+	return size
+end
+
+-- returns a patched object from string `base` according to `delta` data
+local function patch_object(base, delta, base_type)
+	-- insert delta codes into temporary file
+	local df = reader(delta)
+
+	-- retrieve original and result size (for checks)
+	local orig_size = delta_size(df)
+	assert(#base == orig_size, fmt('#base(%d) ~= orig_size(%d)', #base, orig_size))
+
+	local result_size = delta_size(df)
+	local size = result_size
+
+	local result = {}
+
+	-- process the delta codes
+	local cmd = df:read(1)
+	while cmd do
+		cmd = ord(cmd)
+		if cmd == 0 then
+			error('unexpected delta code 0')
+		elseif band(cmd, 0x80) ~= 0 then -- copy a selected part of base data
+			local cp_off, cp_size = 0, 0
+			-- retrieve offset
+			if band(cmd, 0x01) ~= 0 then cp_off = ord(df:read(1)) end
+			if band(cmd, 0x02) ~= 0 then cp_off = cp_off + ord(df:read(1))*256 end
+			if band(cmd, 0x04) ~= 0 then cp_off = cp_off + ord(df:read(1))*256^2 end
+			if band(cmd, 0x08) ~= 0 then cp_off = cp_off + ord(df:read(1))*256^3 end
+			-- retrieve size
+			if band(cmd, 0x10) ~= 0 then cp_size = ord(df:read(1)) end
+			if band(cmd, 0x20) ~= 0 then cp_size = cp_size + ord(df:read(1))*256 end
+			if band(cmd, 0x40) ~= 0 then cp_size = cp_size + ord(df:read(1))*256^2 end
+			if cp_size == 0 then cp_size = 0x10000 end
+			if cp_off + cp_size > #base or cp_size > size then break end
+			-- get the data and append it to result
+			local data = base:sub(cp_off + 1, cp_off + cp_size)
+			insert(result, data)
+			size = size - cp_size
+		else -- insert new data
+			if cmd > size then break end
+			local data = df:read(cmd)
+			insert(result, data)
+			size = size - cmd
+		end
+		cmd = df:read(1)
+	end
+
+	df:close()
+
+	result = concat(result)
+	assert(#result == result_size, fmt('#result(%d) ~= result_size(%d)', #result, result_size))
+	return result, result_size, base_type
+end
+
+Pack = {}
+Pack.__index = Pack
+
+-- read an object from the current location in pack, or from a specific `offset`
+-- if specified
+function Pack:read_object(offset, ignore_data)
+	local f = self.pack_file
+	if offset then
+		f:seek('set', offset)
+	end
+	local curr_pos = f:seek()
+
+	local len, type = read_object_header(f)
+	if type < 5 then -- commit, tree, blob, tag
+		return unpack_object(f, len, type)
+	elseif type == 6 then -- ofs_delta
+		local offset = read_delta_header(f)
+		local delta_data = uncompress_by_len(f, len)
+		if not ignore_data then
+			-- the offset is negative from the current location
+			local base, base_len, base_type = self:read_object(curr_pos - offset)
+			return patch_object(base, delta_data, base_type)
+		end
+	elseif type == 7 then -- ref_delta
+		local sha = f:read(20)
+		local delta_data = uncompress_by_len(f, len)
+		if not ignore_data then
+			-- lookup the object in the pack by sha
+			-- FIXME: maybe lookup in repo/other packs
+			local base_offset = self.index[binary_sha(sha)]
+			local base, base_len, base_type = self:read_object(base_offset)
+			return patch_object(base, delta_data, base_type)
+		end
+	else
+		error('unknown object type: '..type)
+	end
+end
+
+-- returns true if this pack contains the given object
+function Pack:has_object(sha)
+	return self.index[binary_sha(sha)] ~= nil
+end
+
+-- if the object name `sha` exists in the pack, returns a temporary file with the
+-- object content, length and type, otherwise returns nil
+function Pack:get_object(sha)
+	local offset = self.index[binary_sha(sha)]
+	if not offset then
+		print('!!! Failed to find object', readable_sha(sha))
+	end
+
+	local data, len, type = self:read_object(offset)
+	print(readable_sha(sha), len, type, data)
+	local f = tmpfile()
+	f:write(data)
+	f:seek('set', 0)
+
+	return f, len, types[type]
+end
+
+function Pack:unpack(repo)
+	for i=1, self.nobjects do
+		local offset = self.offsets[i]
+		local data, len, type = self:read_object(offset)
+		repo:store_object(data, len, types[type])
+	end
+end
+
+-- parses the index
+function Pack:parse_index(index_file)
+	local f = index_file
+
+	local head = f:read(4)
+	assert(head == '\255tOc', "Incorrect header: " .. head)
+	local version = read_int(f)
+	assert(version == 2, "Incorrect version: " .. version)
+
+	-- first the fanout table (how many objects are in the index, whose
+	-- first byte is below or equal to i)
+	local fanout = {}
+	for i=0, 255 do
+		local nobjs = read_int(f)
+		fanout[i] = nobjs
+	end
+
+	-- the last element in fanout is the number of all objects in index
+	local count = fanout[255]
+
+	-- then come the sorted object names (=sha hash)
+	local tmp = {}
+	for i=1,count do
+		local sha = f:read(20)
+		tmp[i] = { sha = sha }
+	end
+
+	-- then the CRCs (assume ok, skip them)
+	for i=1, count do
+		local crc = f:read(4)
+	end
+
+	-- then come the offsets - read just the 32bit ones, does not handle packs > 2G
+	for i=1, count do
+		local offset = read_int(f)
+		tmp[i].offset = offset
+	end
+
+	-- construct the lookup table
+	local lookup = {}
+	for i=1, count do
+		lookup[tmp[i].sha] = tmp[i].offset
+	end
+	self.index = lookup
+end
+
+-- constructs the index/offsets if the index file is missing
+function Pack:construct_index(path)
+	local index = {}
+	for i=1, self.nobjects do
+		local offset = self.offsets[i]
+		local data, len, type = self:read_object(offset)
+		local sha = object_sha(data, len, types[type])
+		index[binary_sha(sha)] = offset
+	end
+	self.index = index
+end
+
+function Pack:close()
+	self.pack_file:close()
+end
+
+function Pack.open(path)
+	local fp = assert(io.open(path, 'rb')) -- stays open
+	
+	-- read the pack header
+	local head = fp:read(4)
+	assert(head == 'PACK', "Incorrect header: " .. head)
+	local version = read_int(fp)
+	assert(version == 2, "Incorrect version: " .. version)
+	local nobj = read_int(fp)
+
+	local pack = setmetatable({
+		offsets = {},
+		nobjects = nobj,
+		pack_file = fp,
+	}, Pack)
+	
+	-- fill the offsets by traversing through the pack
+	for i=1,nobj do
+		pack.offsets[i] = fp:seek()
+		-- ignore the object data, we only need the offset in the pack
+		pack:read_object(nil, true)
+	end
+
+	-- read the index
+	local fi = io.open((path:gsub('%.pack$', '.idx')), 'rb')
+	if fi then
+		pack:parse_index(fi)
+		fi:close()
+	else
+		pack:construct_index(path)
+	end
+
+	return pack
+end
+
+return Pack

lualibs/git/protocol.lua

@@ -0,0 +1,188 @@
+local socket = require 'socket'
+local urllib = require 'socket.url'
+local lfs = require 'lfs'
+
+local Repo = git.repo.Repo
+local Pack = git.pack.Pack
+local join_path = git.util.join_path
+local parent_dir = git.util.parent_dir
+local make_dir = git.util.make_dir
+local correct_separators = git.util.correct_separators
+
+local assert, error, getmetatable, io, os, pairs, print, require, string, tonumber =
+	assert, error, getmetatable, io, os, pairs, print, require, string, tonumber
+
+local _VERSION, newproxy = _VERSION, newproxy
+
+module(...)
+
+local GIT_PORT = 9418
+
+local function git_connect(host)
+	local sock = assert(socket.connect(host, GIT_PORT))
+	local gitsocket = {}
+
+	function gitsocket:send(data)
+		if not data then -- flush packet
+			sock:send('0000')
+		else
+			local len = #data + 4
+			len = string.format("%04x", len)
+			assert(sock:send(len .. data))
+		end
+	end
+
+	function gitsocket:receive()
+		local len = assert(sock:receive(4))
+		len = tonumber(len, 16)
+		if len == 0 then return end -- flush packet
+		local data = assert(sock:receive(len - 4))
+		return data
+	end
+
+	function gitsocket:close()
+		sock:close()
+	end
+
+	return gitsocket
+end
+
+local function addFinalizer(object, finalizer)
+	if _VERSION <= "Lua 5.1" then
+		local gc = newproxy(true)
+		getmetatable(gc).__gc = finalizer
+		object.__gc = gc
+	else
+		local mt = getmetatable(object)
+		if mt then mt.__gc = finalizer
+		else setmetatable(object, {__gc = finalizer})
+		end
+	end
+end
+
+local function git_fetch(host, path, repo, head, supress_progress)
+	local s = git_connect(host)
+	s:send('git-upload-pack '..path..'\0host='..host..'\0')
+
+	local refs, refsbyname = {}, {}
+	repeat
+		local ref = s:receive()
+		if ref then
+			local sha, name = ref:sub(1,40), ref:sub(42, -2)
+			refs[sha] = name
+			refsbyname[name] = sha
+		end
+	until not ref
+
+	local wantedSha
+	local headsha = head and refsbyname[head]
+
+	for sha, ref in pairs(refs) do
+		-- we implicitly want this ref
+		local wantObject = true 
+		-- unless we ask for a specific head
+		if headsha then         
+			if sha ~= headsha then
+				wantObject = false
+			else
+				wantedSha = sha
+			end
+		end
+		-- or we already have it
+		if repo and repo:has_object(sha) then
+			wantObject = false
+		end
+		if wantObject then
+			s:send('want '..sha..' multi_ack_detailed side-band-64k ofs-delta\n')
+		end
+	end
+
+	if head and not wantedSha then
+		error("Server does not have "..head)
+	end
+
+	s:send('deepen 1')
+	s:send()
+	while s:receive() do end
+	s:send('done\n')
+	
+	assert(s:receive() == "NAK\n")
+	
+	local packname = os.tmpname() .. '.pack'
+	local packfile = assert(io.open(packname, 'wb'))
+	repeat
+		local got = s:receive()
+		if got then
+			-- get sideband channel, 1=pack data, 2=progress, 3=error
+			local cmd = string.byte(got:sub(1,1))
+			local data = got:sub(2)
+			if cmd == 1 then
+				packfile:write(data)
+			elseif cmd == 2 then
+				if not supress_progress then io.write(data) end
+			else
+				error(data)
+			end
+		end
+	until not got
+
+	packfile:close()
+	s:close()
+
+	local pack = Pack.open(packname)
+	if repo then
+		pack:unpack(repo)
+		repo.isShallow = true
+		if wantedSha then
+			local headfile = correct_separators(join_path(repo.dir, head))
+			assert(make_dir(parent_dir(headfile)))
+			local f = assert(io.open(headfile, 'wb'))
+			f:write(wantedSha)
+			f:close()
+		end
+	end
+
+	addFinalizer(pack, function()
+		os.remove(packname)
+	end)
+
+	return pack, wantedSha
+end
+
+function fetch(url, repo, head, supress_progress)
+	if repo then assert(getmetatable(repo) == Repo, "arg #2 is not a repository") end
+	url = urllib.parse(url)
+	if url.scheme == 'git' then
+		local pack, sha = git_fetch(url.host, url.path, repo, head, supress_progress)
+		return pack, sha
+	else
+		error('unsupported scheme: '..url.scheme)
+	end
+end
+
+function remotes(url)
+	-- TODO: refactor common code
+	url = assert(urllib.parse(url))
+
+	if url.scheme ~= 'git' then
+		error('unsupported scheme: '..url.scheme)
+	end
+
+	local host, path = url.host, url.path
+
+	local s = git_connect(host)
+	s:send('git-upload-pack '..path..'\0host='..host..'\0')
+
+	local remote = {}
+	repeat
+		local ref = s:receive()
+		if ref then
+			local sha, name = ref:sub(1,40), ref:sub(42, -2)
+			remote[name] = sha
+		end
+	until not ref
+
+	s:close()
+
+	return remote
+end

lualibs/git/repo.lua

@@ -0,0 +1,283 @@
+local util = require 'git.util'
+local objects = require 'git.objects'
+local core = require 'git.core'
+local pack = require 'git.pack'
+
+local join_path = util.join_path
+local decompressed = util.decompressed
+local read_until_nul = util.read_until_nul
+local to_hex = util.to_hex
+local object_sha = util.object_sha
+local readable_sha = util.readable_sha
+
+local deflate = core.deflate
+
+local lfs = require 'lfs'
+local assert, error, io, ipairs, print, os, setmetatable, string, table =
+	assert, error, io, ipairs, print, os, setmetatable, string, table
+
+module(...)
+
+Repo = {}
+Repo.__index = Repo
+
+-- retrieves an object identified by `sha` from the repository or its packs
+-- returns a file-like object (supports 'read', 'seek' and 'close'), the size
+-- of the object and its type
+-- errors when the object does not exist
+function Repo:raw_object(sha)
+	-- first, look in 'objects' directory
+	-- first byte of sha is the directory, the rest is name of object file
+	sha = readable_sha(sha)
+	local dir = sha:sub(1,2)
+	local file = sha:sub(3)
+	local path = join_path(self.dir, 'objects', dir, file)
+
+	if not lfs.attributes(path, 'size') then
+		-- then, try to look in packs
+		for _, pack in ipairs(self.packs) do
+			local obj, len, typ = pack:get_object(sha)
+			if obj then
+				return obj, len, typ
+			end
+		end
+		error('Object not found in object neither in packs: '..sha)
+	else		
+		-- the objects are zlib compressed
+		local f = decompressed(path)
+
+		-- retrieve the type and length - <type> SP <len> \0 <data...>
+		local content = read_until_nul(f)
+		local typ, len = content:match('(%w+) (%d+)')
+
+		return f, len, typ
+	end
+end
+
+--- Store a new object into the repository in `objects` directory.
+-- @param data A string containing the contents of the new file.
+-- @param len The length of the data.
+-- @param type One of 'commit', 'blob', 'tree', 'tag'
+function Repo:store_object(data, len, type)
+	local sha = readable_sha(object_sha(data, len, type))
+	local dir = sha:sub(1,2)
+	local file = sha:sub(3)
+	util.make_dir(join_path(self.dir, 'objects', dir))
+	local path = join_path(self.dir, 'objects', dir, file)
+	local fo = assert(io.open(path, 'wb'))
+	local header = type .. ' ' .. len .. '\0'
+	local compressed = deflate()(header .. data, "finish")
+	fo:write(compressed)
+	fo:close()
+end
+
+local function resolvetag(f)
+	local tag
+	local line = f:read()
+	while line do
+		tag = line:match('^object (%x+)$')
+		if tag then break end
+		line = f:read()
+	end
+	f:close()
+	return tag
+end
+
+function Repo:commit(sha)
+	local f, len, typ = self:raw_object(sha)
+	while typ == 'tag' do
+		sha = assert(resolvetag(f), 'could not parse tag for '..readable_sha(sha))
+		f, len, typ = self:raw_object(sha)
+	end
+	assert(typ == 'commit', string.format('%s (%s) is not a commit', sha, typ))
+
+	local commit = { id = sha, repo = self, stored = true, parents = {} }
+	repeat
+		local line = f:read()
+		if not line then break end
+
+		local space = line:find(' ') or 0
+		local word = line:sub(1, space - 1)
+		local afterSpace = line:sub(space + 1)
+
+		if word == 'tree' then
+			commit.tree_sha = afterSpace
+		elseif word == 'parent' then
+			table.insert(commit.parents, afterSpace)
+		elseif word == 'author' then
+			commit.author = afterSpace
+		elseif word == 'committer' then
+			commit.committer = afterSpace
+		elseif commit.message then
+			table.insert(commit.message, line)
+		elseif line == '' then
+			commit.message = {}
+		end
+	until false -- ends with break
+	f:close()
+
+	commit.message = table.concat(commit.message, '\n')
+
+	return setmetatable(commit, objects.Commit)
+end
+
+function Repo:tree(sha)
+	local f, len, typ = self:raw_object(sha)
+	assert(typ == 'tree', string.format('%s (%s) is not a tree', sha, typ))
+
+	local tree = { id = sha, repo = self, stored = true, _entries = {} }
+
+	while true do
+		local info = read_until_nul(f)
+		if not info then break end
+		local entry_sha = to_hex(f:read(20))
+		local mode, name = info:match('^(%d+)%s(.+)$')
+		local entry_type = 'blob'
+		if mode == '40000' then
+			entry_type = 'tree'
+		elseif mode == '160000' then
+			entry_type = 'commit'
+		end
+		tree._entries[name] = { mode = mode, id = entry_sha, type = entry_type }
+	end
+
+	f:close()
+
+	return setmetatable(tree, objects.Tree)
+end
+
+-- retrieves a Blob
+function Repo:blob(sha)
+	local f, len, typ = self:raw_object(sha)
+	f:close() -- can be reopened in Blob:content()
+
+	assert(typ == 'blob', string.format('%s (%s) is not a blob', sha, typ))
+	return setmetatable({ 
+		id = sha,
+		len = len,
+		repo = self,
+		stored = true }, objects.Blob)
+end
+
+function Repo:head()
+	return self:commit(self.refs.HEAD)
+end
+
+function Repo:has_object(sha)
+	local dir = sha:sub(1,2)
+	local file = sha:sub(3)
+	local path = join_path(self.dir, 'objects', dir, file)
+
+	if lfs.attributes(path, 'size') then return true end
+
+	for _, pack in ipairs(self.packs) do
+		local has = pack:has_object(sha)
+		if has then return true end
+	end
+
+	return false
+end
+
+function Repo:checkout(sha, target)
+	if not target then target = self.workDir end
+	assert(target, 'target directory not specified')
+
+	local commit = self:commit(sha)
+	commit:checkout(target)
+
+	-- if the repo was checked out using the deepen command (one level of history only)
+	-- mark the commit's parent as shalow, that is it has no history
+	if self.isShallow then
+		-- if it has a parent, mark it shallow
+		if commit.parents[1] then
+			local f = assert(io.open(self.dir .. '/shallow', "w"))
+			f:write(commit.parents[1], '\n')
+			f:close()
+		end
+	end
+end
+
+function Repo:close()
+	for _, pack in ipairs(self.packs) do
+		pack:close()
+	end
+end	
+
+function create(dir)
+	if not dir:match('%.git.?$') then
+		dir = join_path(dir, '.git')
+	end
+	
+	util.make_dir(dir)
+	util.make_dir(dir .. '/branches')
+	util.make_dir(dir .. '/hooks')
+	util.make_dir(dir .. '/info')	
+	util.make_dir(dir .. '/objects/info')
+	util.make_dir(dir .. '/objects/pack')
+	util.make_dir(dir .. '/refs/heads')
+	util.make_dir(dir .. '/refs/tags')
+	util.make_dir(dir .. '/refs/remotes')
+
+	do
+		local f = assert(io.open(dir .. "/HEAD", "w"))
+		f:write("ref: refs/heads/master\n")
+		f:close()
+	end
+
+	local refs = {}
+	local packs = {}
+
+	return setmetatable({
+		dir = dir,
+		refs = refs,
+		packs = packs,
+	}, Repo)
+end
+
+-- opens a repository located in working directory `dir` or directly a .git repo
+function open(dir)
+	local workDir = dir
+	if not dir:match('%.git.?$') then
+		dir = join_path(dir, '.git')
+	else
+		workDir = nil -- no working directory, working directly with repo
+	end
+
+	local refs = {}
+	for _,d in ipairs{'refs/heads', 'refs/tags'} do
+		for fn in lfs.dir(join_path(dir, d)) do
+			if fn ~= '.' and fn ~= '..' then
+				local path = join_path(dir, d, fn)
+				local f = assert(io.open(path), 'rb')
+				local ref = f:read()
+				refs[join_path(d, fn)] = ref
+				f:close()
+			end
+		end
+	end
+
+	local packs = {}
+	for fn in lfs.dir(join_path(dir, 'objects/pack')) do
+		if fn:match('%.pack$') then
+			local path = join_path(dir, 'objects/pack', fn)
+			table.insert(packs, pack.open(path))
+		end
+	end
+
+	local head = io.open(join_path(dir, 'HEAD'), 'rb')
+	if head then
+		local src = head:read()
+		local HEAD = src:match('ref: (.-)$')
+		refs.HEAD = refs[HEAD]
+		head:close()
+	end
+
+	return setmetatable({
+		dir = dir,
+		workDir = workDir,
+		refs = refs,
+		packs = packs,
+	}, Repo)
+end
+
+return Repo

lualibs/git/util.lua

@@ -0,0 +1,233 @@
+local lfs = require 'lfs'
+local core = require 'git.core'
+local deflate = core.deflate
+local inflate = core.inflate
+local sha = core.sha
+
+module(..., package.seeall)
+
+local BUF_SIZE = 4096
+
+local dirsep = package.config:sub(1,1)
+
+-- replaces '/' path separators on Windows with the correct ones ('\\')
+function correct_separators(path)
+	return path:gsub('/', dirsep)
+end
+
+-- joins several path components into a single path, uses system-specific directory
+-- separator, cleans input, i.e. join_path('a/', 'b', 'c/') => 'a/b/c'
+function join_path(...)
+	local n = select('#', ...)
+	local args = {...}
+	for i=1,n do
+		args[i] = args[i]:gsub(dirsep..'?$', '')
+	end
+	return table.concat(args, dirsep, 1, n)
+end
+
+-- Return the path with the all occurences of '/.' or '\.' (representing
+-- the current directory) removed.
+local function remove_curr_dir_dots(path)
+	while path:match(dirsep .. "%." .. dirsep) do             -- match("/%./")
+		path = path:gsub(dirsep .. "%." .. dirsep, dirsep)    -- gsub("/%./", "/")
+	end
+	return path:gsub(dirsep .. "%.$", "")                     -- gsub("/%.$", "")
+end
+
+-- Return whether the path is a root.
+local function is_root(path)
+	return path:find("^[%u%U.]?:?[/\\]$")
+end
+
+-- Return the path with the unnecessary trailing separator removed.
+local function remove_trailing(path)
+	if path:sub(-1) == dirsep and not is_root(path) then path = path:sub(1,-2) end
+	return path
+end
+
+-- Extract file or directory name from its path.
+local function extract_name(path)
+	if is_root(path) then return path end
+
+	path = remove_trailing(path)
+	path = path:gsub("^.*" .. dirsep, "")
+	return path
+end
+
+-- Return the string 'str', with all magic (pattern) characters escaped.
+local function escape_magic(str)
+	local escaped = str:gsub('[%-%.%+%[%]%(%)%^%%%?%*%^%$]','%%%1')
+	return escaped
+end
+
+-- Return parent directory of the 'path' or nil if there's no parent directory.
+-- If 'path' is a path to file, return the directory the file is in.
+function parent_dir(path)
+	path = remove_curr_dir_dots(path)
+	path = remove_trailing(path)
+
+	local dir = path:gsub(escape_magic(extract_name(path)) .. "$", "")
+	if dir == "" then
+		return nil
+	else
+		return remove_trailing(dir)
+	end
+end
+
+-- Make a new directory, making also all of its parent directories that doesn't exist.
+function make_dir(path)
+	if lfs.attributes(path) then
+		return true
+	else
+		local par_dir = parent_dir(path)
+		if par_dir then
+			assert(make_dir(par_dir))
+		end
+		return lfs.mkdir(path)
+	end
+end
+
+
+-- Reader class
+-- adapted from Penlight: https://raw.github.com/stevedonovan/Penlight/master/lua/pl/stringio.lua
+ 
+local SR = {}
+SR.__index = SR
+
+function SR:_read(fmt)
+    local i,str = self.i,self.str
+    local sz = #str
+    if i > sz then return nil, "past end of file" end
+    local res
+    if fmt == '*l' or fmt == '*L' then
+        local idx = str:find('\n',i) or (sz+1)
+        res = str:sub(i,fmt == '*l' and idx-1 or idx)
+        self.i = idx+1
+    elseif fmt == '*a' then
+        res = str:sub(i)
+        self.i = sz+1
+    elseif fmt == '*n' then
+        local _,i2,i2,idx
+        _,idx = str:find ('%s*%d+',i)
+        _,i2 = str:find ('^%.%d+',idx+1)
+        if i2 then idx = i2 end
+        _,i2 = str:find ('^[eE][%+%-]*%d+',idx+1)
+        if i2 then idx = i2 end
+        local val = str:sub(i,idx)
+        res = tonumber(val)
+        self.i = idx+1
+    elseif type(fmt) == 'number' then
+        res = str:sub(i,i+fmt-1)
+        self.i = i + fmt
+    else
+        error("bad read format",2)
+    end
+    return res
+		end
+
+function SR:read(...)
+    if select('#',...) == 0 then
+        return self:_read('*l')
+    else
+        local res, fmts = {},{...}
+        for i = 1, #fmts do
+            res[i] = self:_read(fmts[i])
+			end
+        return unpack(res)
+				end
+			end
+
+function SR:seek(whence,offset)
+    local base
+    whence = whence or 'cur'
+    offset = offset or 0
+    if whence == 'set' then
+        base = 1
+    elseif whence == 'cur' then
+        base = self.i
+    elseif whence == 'end' then
+        base = #self.str
+	end
+    self.i = base + offset
+    return self.i
+end
+
+function SR:close() -- for compatibility only
+end
+
+--- create a file-like object for reading from a given string.
+-- @param s The input string.
+function reader(s)
+    return setmetatable({str=s,i=1},SR)
+end
+
+
+-- decompress the file and return a handle to temporary uncompressed file
+function decompressed(path)
+	local fi = assert(io.open(path, 'rb'))
+	local result = {}
+
+	local z = inflate()
+	repeat
+		local str = fi:read(BUF_SIZE)
+		local data = z(str)
+		if type(data) == 'string' then
+			result[#result+1] = data
+		else print('!!!', data) end
+	until not str
+	fi:close()
+
+	return reader(table.concat(result))
+end
+
+-- reads until the byte \0, consumes it and returns the string up to the \0
+function read_until_nul(f)
+	local t = {}
+	repeat
+		local c = f:read(1)
+		if c and c ~= '\0' then t[#t+1] = c end
+	until not c or c == '\0'
+	if #t > 0 then
+		return table.concat(t)
+	else
+		return nil
+	end
+end
+
+-- converts a string to lowercase hex
+function to_hex(s)
+	return (s:gsub('.', function(c)
+		return string.format('%02x', string.byte(c))
+	end))
+end
+
+-- converts a string from hex to binary
+function from_hex(s)
+	return (s:gsub('..', function(cc)
+		return string.char(tonumber(cc, 16))
+	end))
+end
+
+-- always returns readable (hex) hash
+function readable_sha(s)
+	if #s ~= 40 then return to_hex(s)
+	else return s end
+end
+
+-- always returns binary hash
+function binary_sha(s)
+	if #s ~= 20 then return from_hex(s)
+	else return s end
+end
+
+function object_sha(data, len, type)
+	local header = type .. ' ' .. len .. '\0'
+	local res = sha(header .. data)
+	return res
+end
+
+function deflate(data)
+	local c = deflate()
+	return c(data, "finish")
+end

lualibs/ltn12.lua

@@ -2,7 +2,6 @@
 -- LTN12 - Filters, sources, sinks and pumps.
 -- LuaSocket toolkit.
 -- Author: Diego Nehab
--- RCS ID: $Id: ltn12.lua 1418 2006-04-25 09:38:15Z 3rdparty $
 -----------------------------------------------------------------------------
 
 -----------------------------------------------------------------------------
@@ -11,15 +10,20 @@
 local string = require("string")
 local table = require("table")
 local base = _G
-module("ltn12")
+local _M = {}
+if module then -- heuristic for exporting a global package table
+    ltn12 = _M
+end
+local filter,source,sink,pump = {},{},{},{}
 
-filter = {}
-source = {}
-sink = {}
-pump = {}
+_M.filter = filter
+_M.source = source
+_M.sink = sink
+_M.pump = pump
 
 -- 2048 seems to be better in windows...
-BLOCKSIZE = 2048
+_M.BLOCKSIZE = 2048
+_M._VERSION = "LTN12 1.0.3"
 
 -----------------------------------------------------------------------------
 -- Filter stuff
@@ -37,7 +41,8 @@ end
 -- chains a bunch of filters together
 -- (thanks to Wim Couwenberg)
 function filter.chain(...)
-    local n = table.getn(arg)
+    local arg = {...}
+    local n = select('#',...)
     local top, index = 1, 1
     local retry = ""
     return function(chunk)
@@ -88,7 +93,7 @@ end
 function source.file(handle, io_err)
     if handle then
         return function()
-            local chunk = handle:read(BLOCKSIZE)
+            local chunk = handle:read(_M.BLOCKSIZE)
             if not chunk then handle:close() end
             return chunk
         end
@@ -111,8 +116,8 @@ function source.string(s)
     if s then
         local i = 1
         return function()
-            local chunk = string.sub(s, i, i+BLOCKSIZE-1)
-            i = i + BLOCKSIZE
+            local chunk = string.sub(s, i, i+_M.BLOCKSIZE-1)
+            i = i + _M.BLOCKSIZE
             if chunk ~= "" then return chunk
             else return nil end
         end
@@ -185,6 +190,7 @@ end
 -- other, as if they were concatenated
 -- (thanks to Wim Couwenberg)
 function source.cat(...)
+    local arg = {...}
     local src = table.remove(arg, 1)
     return function()
         while src do
@@ -289,3 +295,4 @@ function pump.all(src, snk, step)
     end
 end
 
+return _M

lualibs/luadist.lua

@@ -0,0 +1,746 @@
+#!/usr/bin/env lua
+
+-- Command line interface to LuaDist-git.
+
+local dist = require "dist"
+local utils = require "dist.utils"
+local depends = require "dist.depends"
+local package = require "dist.package"
+local mf = require "dist.manifest"
+local cfg = require "dist.config"
+local sys = require "dist.sys"
+
+-- CLI commands of Luadist.
+local commands
+commands = {
+
+    -- Print help for this command line interface.
+    ["help"] = {
+        help = [[
+Usage: luadist [DEPLOYMENT_DIRECTORY] <COMMAND> [ARGUMENTS...] [-VARIABLES...]
+
+    Commands:
+
+        help      - print this help
+        install   - install modules
+        remove    - remove modules
+        refresh   - update information about modules in repositories
+        list      - list installed modules
+        info      - show information about modules
+        search    - search repositories for modules
+        fetch     - download modules
+        make      - manually deploy modules from local paths
+        upload    - upload installed modules to their repositories
+        tree      - print dependency tree of a module
+        selftest  - run the selftest of LuaDist
+
+    To get help on specific command, run:
+
+        luadist help <COMMAND>
+        ]],
+        run = function (deploy_dir, help_item)
+            deploy_dir = deploy_dir or dist.get_deploy_dir()
+            help_item = help_item or {}
+            assert(type(deploy_dir) == "string", "luadist.help: Argument 'deploy_dir' is not a string.")
+            assert(type(help_item) == "table", "luadist.help: Argument 'help_item' is not a table.")
+            deploy_dir = sys.abs_path(deploy_dir)
+
+            if not help_item or not commands[help_item[1]] then
+                help_item = "help"
+            else
+                help_item = help_item[1]
+            end
+
+            print_info()
+            print(commands[help_item].help)
+            return 0
+        end
+    },
+
+    -- Install modules.
+    ["install"] = {
+        help = [[
+Usage: luadist [DEPLOYMENT_DIRECTORY] install MODULES... [-VARIABLES...]
+
+    The 'install' command will install specified MODULES to
+    DEPLOYMENT_DIRECTORY. LuaDist will also automatically resolve, download
+    and install all dependencies.
+
+    If DEPLOYMENT_DIRECTORY is not specified, the deployment directory
+    of LuaDist is used.
+
+    You can use * (an asterisk sign) in the name of the module as a wildcard
+    with the meaning 'any symbols' (in most shells, the module name then must
+    be quoted to prevent the expansion of asterisk by the shell itself).
+
+    Optional CMake VARIABLES in -D format (e.g. -Dvariable=value) or LuaDist
+    configuration VARIABLES (e.g. -variable=value) can be specified.
+
+    The -simulate configuration option makes LuaDist only to simulate the
+    installation of modules (no modules will be really installed).
+        ]],
+
+        run = function (deploy_dir, modules, cmake_variables)
+            deploy_dir = deploy_dir or dist.get_deploy_dir()
+            if type(modules) == "string" then modules = {modules} end
+            cmake_variables = cmake_variables or {}
+            assert(type(deploy_dir) == "string", "luadist.install: Argument 'deploy_dir' is not a string.")
+            assert(type(modules) == "table", "luadist.install: Argument 'modules' is not a string or table.")
+            assert(type(cmake_variables) == "table", "luadist.install: Argument 'cmake_variables' is not a table.")
+            deploy_dir = sys.abs_path(deploy_dir)
+
+            if cfg.simulate then
+                print("NOTE: this is just simulation.")
+            end
+
+            if #modules == 0 then
+                print("No modules to install specified.")
+                return 0
+            end
+
+            local ok, err = dist.install(modules, deploy_dir, cmake_variables)
+            if not ok then
+                print(err)
+                os.exit(1)
+            else
+               print((cfg.simulate and "Simulated installation" or "Installation") .. " successful.")
+               return 0
+            end
+        end
+    },
+
+    -- Remove modules.
+    ["remove"] = {
+        help = [[
+Usage: luadist [DEPLOYMENT_DIRECTORY] remove MODULES... [-VARIABLES...]
+
+    The 'remove' command will remove specified MODULES from
+    DEPLOYMENT_DIRECTORY. If no module is specified, all modules
+    will be removed.
+
+    If DEPLOYMENT_DIRECTORY is not specified, the deployment directory
+    of LuaDist is used. If no MODULES are specified, all installed modules
+    will be removed.
+
+    You can use * (an asterisk sign) in the name of the module as a wildcard
+    with the meaning 'any symbols' (in most shells, the module name then must
+    be quoted to prevent the expansion of asterisk by the shell itself).
+
+    Optional LuaDist configuration VARIABLES (e.g. -variable=value) can be
+    specified.
+
+    WARNING: dependencies between modules are NOT taken into account when
+    removing modules!
+        ]],
+
+        run = function (deploy_dir, modules)
+            deploy_dir = deploy_dir or dist.get_deploy_dir()
+            if type(modules) == "string" then modules = {modules} end
+            assert(type(deploy_dir) == "string", "luadist.remove: Argument 'deploy_dir' is not a string.")
+            assert(type(modules) == "table", "luadist.remove: Argument 'modules' is not a string or table.")
+            deploy_dir = sys.abs_path(deploy_dir)
+
+            local num, err = dist.remove(modules, deploy_dir)
+            if not num then
+                print(err)
+                os.exit(1)
+            else
+               print("Removed modules: " .. num)
+               return 0
+            end
+        end
+    },
+
+    -- Update repositories.
+    ["refresh"] = {
+        help = [[
+Usage: luadist [DEPLOYMENT_DIRECTORY] refresh [-VARIABLES...]
+
+    The 'refresh' command will update information about modules in all software
+    repositories of specified DEPLOYMENT_DIRECTORY. Also, the cached dependency
+    manifest, built from previous installations or invocations of 'tree'
+    functionality will be deleted.
+
+    If DEPLOYMENT_DIRECTORY is not specified, the deployment directory
+    of LuaDist is used.
+
+    Optional LuaDist configuration VARIABLES (e.g. -variable=value) can be
+    specified.
+        ]],
+
+        run = function (deploy_dir)
+            deploy_dir = deploy_dir or dist.get_deploy_dir()
+            assert(type(deploy_dir) == "string", "luadist.refresh: Argument 'deploy_dir' is not a string.")
+            deploy_dir = sys.abs_path(deploy_dir)
+
+            -- TODO: should be deleting the dep_manifest decoupled from refreshing the repository info?
+            -- delete cached dependency manifest
+            local dep_manifest_file = sys.abs_path(sys.make_path(deploy_dir, cfg.dep_cache_file))
+            local dep_mf_deleted = false
+            if sys.exists(dep_manifest_file) then
+                sys.delete(dep_manifest_file)
+                dep_mf_deleted = true
+            end
+
+            -- refresh repository information
+            local ok, err = dist.update_manifest(deploy_dir)
+            if not ok then
+                print(err)
+                os.exit(1)
+            else
+               print("Repositories successfuly updated" .. (dep_mf_deleted and " and dependency cache deleted" or "") .. ".")
+               return 0
+            end
+        end
+    },
+
+    -- Manually deploy modules.
+    ["make"] = {
+        help = [[
+Usage: luadist [DEPLOYMENT_DIRECTORY] make MODULE_PATHS... [-VARIABLES...]
+
+    The 'make' command will manually deploy modules from specified local
+    MODULE_PATHS into the DEPLOYMENT_DIRECTORY.
+
+    The MODULE_PATHS will be preserved. If DEPLOYMENT_DIRECTORY is not
+    specified, the deployment directory of LuaDist is used.
+
+    Optional CMake VARIABLES in -D format (e.g. -Dvariable=value) or LuaDist
+    configuration VARIABLES (e.g. -variable=value) can be specified.
+
+    The -simulate configuration option makes LuaDist only to simulate the
+    deployment of modules (no modules will be really deployed).
+
+    WARNING: this command does NOT check whether the dependencies of deployed
+    modules are satisfied or not!
+        ]],
+
+        run = function (deploy_dir, module_paths, cmake_variables)
+            deploy_dir = deploy_dir or dist.get_deploy_dir()
+            module_paths = module_paths or {}
+            cmake_variables = cmake_variables or {}
+            assert(type(deploy_dir) == "string", "luadist.make: Argument 'deploy_dir' is not a string.")
+            assert(type(module_paths) == "table", "luadist.make: Argument 'module_paths' is not a table.")
+            assert(type(cmake_variables) == "table", "luadist.make: Argument 'cmake_variables' is not a table.")
+            deploy_dir = sys.abs_path(deploy_dir)
+
+            if cfg.simulate then
+                print("NOTE: this is just simulation.")
+            end
+
+            if #module_paths == 0 then
+                print("No module paths to deploy specified.")
+                return 0
+            end
+
+            local ok, err = dist.make(deploy_dir, module_paths, cmake_variables)
+            if not ok then
+                print(err)
+                os.exit(1)
+            end
+            print((cfg.simulate and "Simulated deployment" or "Deployment") .. " successful.")
+            return 0
+        end
+    },
+
+    -- Download modules.
+    ["fetch"] = {
+        help = [[
+Usage: luadist [FETCH_DIRECTORY] fetch MODULES... [-VARIABLES...]
+
+    The 'fetch' command will download specified MODULES to the FETCH_DIRECTORY.
+
+    If no FETCH_DIRECTORY is specified, the temporary directory of LuaDist
+    deployment directory (i.e. ']] .. cfg.temp_dir .. [[') is used.
+    If the version is not specified in module name, the most recent version
+    available will be downloaded.
+
+    Optional LuaDist configuration VARIABLES (e.g. -variable=value) can be
+    specified.
+        ]],
+
+        run = function (fetch_dir, modules)
+            fetch_dir = fetch_dir or dist.get_deploy_dir()
+            modules = modules or {}
+            assert(type(fetch_dir) == "string", "luadist.fetch: Argument 'fetch_dir' is not a string.")
+            assert(type(modules) == "table", "luadist.fetch: Argument 'modules' is not a table.")
+            fetch_dir = sys.abs_path(fetch_dir)
+
+            -- if the default parameter (i.e. deploy_dir) is passed, use the default temp_dir
+            if fetch_dir == dist.get_deploy_dir() then
+                fetch_dir = sys.make_path(fetch_dir, cfg.temp_dir)
+            end
+
+            if #modules == 0 then
+                print("No modules to download specified.")
+                return 0
+            end
+
+            local ok, err = dist.fetch(modules, fetch_dir)
+            if not ok then
+                print(err)
+                os.exit(1)
+            else
+                print("Modules successfuly downloaded to '" .. fetch_dir .. "'.")
+                return 0
+            end
+        end
+    },
+
+    -- Upload modules.
+    ["upload"] = {
+        help = [[
+Usage: luadist [DEPLOYMENT_DIRECTORY] upload MODULES... [-VARIABLES...]
+
+    The 'upload' command will upload the binary versions of specified MODULES,
+    installed in the DEPLOYMENT_DIRECTORY, to their LuaDist repositories.
+
+    Base url of repositories is given by configuration variable 'upload_url'
+    (by default ']] .. cfg.upload_url .. [[') which you can change.
+    E.g.: Binary version of module 'lua', installed in DEPLOYMENT_DIRECTORY,
+    will now be uploaded to repository ']] .. cfg.upload_url .. [[lua.git'.
+
+    Organization of uploaded modules and their repositories is subject
+    to the conventions described in more detail in the source code
+    of the 'dist.upload_modules()' function (file 'dist/init.lua').
+
+    If DEPLOYMENT_DIRECTORY is not specified, the deployment directory
+    of LuaDist is used. If no MODULES are specified, all installed modules
+    will be uploaded.
+
+    You can use * (an asterisk sign) in the name of the module as a wildcard
+    with the meaning 'any symbols' (in most shells, the module name then must
+    be quoted to prevent the expansion of asterisk by the shell itself).
+
+    Optional LuaDist configuration VARIABLES (e.g. -variable=value) can be
+    specified.
+        ]],
+
+        run = function (deploy_dir, modules)
+            -- check if we have git
+            local ok = utils.system_dependency_available("git", "git --version")
+            if not ok then os.exit(1) end
+
+            deploy_dir = deploy_dir or dist.get_deploy_dir()
+            if type(modules) == "string" then modules = {modules} end
+            assert(type(deploy_dir) == "string", "luadist.upload: Argument 'deploy_dir' is not a string.")
+            assert(type(modules) == "table", "luadist.upload: Argument 'modules' is not a string or table.")
+            deploy_dir = sys.abs_path(deploy_dir)
+
+            local num, err = dist.upload_modules(deploy_dir, modules, cfg.upload_url)
+            if not num then
+                print(err)
+                os.exit(1)
+            else
+               print("Uploaded modules: " .. num)
+               return 0
+            end
+        end
+    },
+
+    -- List installed modules.
+    ["list"] = {
+        help = [[
+Usage: luadist [DEPLOYMENT_DIRECTORY] list [STRINGS...] [-VARIABLES...]
+
+    The 'list' command will list all modules installed in specified
+    DEPLOYMENT_DIRECTORY, which contain one or more optional STRINGS.
+
+    If DEPLOYMENT_DIRECTORY is not specified, the deployment directory
+    of LuaDist is used. If STRINGS are not specified, all installed modules
+    are listed.
+
+    Optional LuaDist configuration VARIABLES (e.g. -variable=value) can be
+    specified.
+        ]],
+
+        run = function (deploy_dir, strings)
+            deploy_dir = deploy_dir or dist.get_deploy_dir()
+            strings = strings or {}
+            assert(type(deploy_dir) == "string", "luadist.list: Argument 'deploy_dir' is not a string.")
+            assert(type(strings) == "table", "luadist.list: Argument 'strings' is not a table.")
+            deploy_dir = sys.abs_path(deploy_dir)
+
+            local deployed = dist.get_deployed(deploy_dir)
+            deployed  = depends.filter_packages_by_strings(deployed, strings)
+
+            print("\nInstalled modules:")
+            print("==================\n")
+            for _, pkg in pairs(deployed) do
+                print("  " .. pkg.name .. "-" .. pkg.version .. "\t(" .. pkg.arch .. "-" .. pkg.type .. ")" .. (pkg.provided_by and "\t [provided by " .. pkg.provided_by .. "]" or ""))
+            end
+            print()
+            return 0
+        end
+    },
+
+    -- Search for modules in repositories.
+    ["search"] = {
+        help = [[
+Usage: luadist [DEPLOYMENT_DIRECTORY] search [STRINGS...] [-VARIABLES...]
+
+    The 'search' command will list all modules from repositories, which contain
+    one or more STRINGS.
+
+    If no STRINGS are specified, all available modules are listed.
+
+    Optional LuaDist configuration VARIABLES (e.g. -variable=value) can be
+    specified.
+        ]],
+
+        run = function (deploy_dir, strings)
+            deploy_dir = deploy_dir or dist.get_deploy_dir()
+            strings = strings or {}
+            assert(type(deploy_dir) == "string", "luadist.search: Argument 'deploy_dir' is not a string.")
+            assert(type(strings) == "table", "luadist.search: Argument 'strings' is not a table.")
+            deploy_dir = sys.abs_path(deploy_dir)
+
+            local available, err = mf.get_manifest()
+            if not available then
+                print(err)
+                os.exit(1)
+            end
+
+            available = depends.filter_packages_by_strings(available, strings)
+            available = depends.sort_by_names(available)
+
+            print("\nModules found:")
+            print("==============\n")
+            for _, pkg in pairs(available) do
+                print("  " .. pkg.name)
+            end
+            print()
+            return 0
+        end
+    },
+
+    -- Show information about modules.
+    ["info"] = {
+        help = [[
+Usage: luadist [DEPLOYMENT_DIRECTORY] info [MODULES...] [-VARIABLES...]
+
+    The 'info' command shows information about specified modules from
+    repositories. This command also shows whether modules are installed
+    in DEPLOYMENT_DIRECTORY.
+
+    If no MODULES are specified, all available modules are shown.
+    If DEPLOYMENT_DIRECTORY is not specified, the deployment directory
+    of LuaDist is used.
+
+    Optional LuaDist configuration VARIABLES (e.g. -variable=value) can be
+    specified.
+        ]],
+
+        run = function (deploy_dir, modules)
+            deploy_dir = deploy_dir or dist.get_deploy_dir()
+            modules = modules or {}
+            assert(type(deploy_dir) == "string", "luadist.info: Argument 'deploy_dir' is not a string.")
+            assert(type(modules) == "table", "luadist.info: Argument 'modules' is not a table.")
+            deploy_dir = sys.abs_path(deploy_dir)
+
+            local manifest, err = mf.get_manifest()
+            if not manifest then
+                print(err)
+                os.exit(1)
+            end
+
+            -- if no packages specified explicitly, show just info from .gitmodules for all packages available
+            if #modules == 0 then
+
+                modules = manifest
+                modules = depends.sort_by_names(modules)
+                local deployed = dist.get_deployed(deploy_dir)
+
+                print("")
+                for _, pkg in pairs(modules) do
+                    print("  " .. pkg.name)
+                    print("  Repository url: " .. (pkg.path or "N/A"))
+                    print()
+                end
+                return 0
+
+            -- if some packages explicitly specified, retrieve and show detailed info about them
+            else
+
+                if #modules > 5 then
+                    print("NOTE: More than 5 modules specified - operation may take a longer time.")
+                end
+
+                local deployed = dist.get_deployed(deploy_dir)
+
+                for _, module in pairs(modules) do
+                    manifest, err = package.get_versions_info(module, manifest, deploy_dir, deployed)
+                    if not manifest then
+                        print(err)
+                        os.exit(1)
+                    end
+                end
+
+                modules = depends.find_packages(modules, manifest)
+                modules = depends.sort_by_names(modules)
+
+                print("")
+                for _, pkg in pairs(modules) do
+                    print("  " .. pkg.name .. "-" .. pkg.version .. "  (" .. pkg.arch .. "-" .. pkg.type ..")" .. (pkg.from_installed and "  [info taken from installed version]" or ""))
+                    print("  Description: " .. (pkg.desc or "N/A"))
+                    print("  Author: " .. (pkg.author or "N/A"))
+                    print("  Homepage: " .. (pkg.url or "N/A"))
+                    print("  License: " .. (pkg.license or "N/A"))
+                    print("  Repository url: " .. (pkg.path or "N/A"))
+                    print("  Maintainer: " .. (pkg.maintainer or "N/A"))
+                    if pkg.provides then print("  Provides: " .. utils.table_tostring(pkg.provides)) end
+                    if pkg.depends then print("  Depends: " .. utils.table_tostring(pkg.depends)) end
+                    if pkg.conflicts then print("  Conflicts: " .. utils.table_tostring(pkg.conflicts)) end
+                    print("  State: " .. (depends.is_installed(pkg.name, deployed, pkg.version) and "installed" or "not installed"))
+                    print()
+                end
+                return 0
+            end
+
+        end
+    },
+
+    -- Print dependency tree.
+    ["tree"] = {
+        help = [[
+Usage: luadist [DEPLOYMENT_DIRECTORY] tree [MODULES...] [-VARIABLES...]
+
+    The 'tree' command prints dependency tree for specified modules.
+
+    If no MODULES are specified, trees for all available modules are printed.
+    This information about modules is being cached in dependency manifest.
+
+    Optional LuaDist configuration VARIABLES (e.g. -variable=value) can be
+    specified.
+        ]],
+
+        run = function (deploy_dir, modules)
+            deploy_dir = deploy_dir or dist.get_deploy_dir()
+            modules = modules or {}
+            assert(type(deploy_dir) == "string", "luadist.info: Argument 'deploy_dir' is not a string.")
+            assert(type(modules) == "table", "luadist.info: Argument 'modules' is not a table.")
+            deploy_dir = sys.abs_path(deploy_dir)
+
+            local manifest, err = mf.get_manifest()
+            if not manifest then
+                print(err)
+                os.exit(1)
+            end
+
+            -- if no modules specified explicitly, assume all modules
+            if #modules == 0 then modules = depends.sort_by_names(manifest) end
+            print("Getting dependency information... (this may take a lot of time)")
+
+            for _, module in pairs(modules) do
+
+                -- if all modules are being queried, extract the name
+                if type(module) == "table" then module = module.name end
+
+                local dep_manifest, err = dist.dependency_info(module, deploy_dir)
+                if not dep_manifest then
+                    print(err)
+                    os.exit(1)
+                else
+
+                    -- print the dependency tree
+                    local heading = "Dependency tree for '" .. module .. "' (on " .. cfg.arch .. "-" .. cfg.type .. "):"
+                    print("\n" .. heading .. "")
+                    print(string.rep("=", #heading) .. "\n")
+
+                    for _, pkg in pairs(dep_manifest) do
+
+                        local pkg_version, pkg_tag = pkg.version, pkg.version
+                        if pkg.was_scm_version then
+                            pkg_version, pkg_tag = "scm", "HEAD"
+                        end
+                        print("  " .. pkg.name .. "-" .. pkg_version .. " (" .. pkg.path .. ", " .. pkg_tag .. ")")
+                        if pkg.depends then
+                            for _, dep in pairs(pkg.depends) do
+                                if type(dep) ~= "table" then
+                                    local found = depends.sort_by_versions(depends.find_packages(dep, dep_manifest))[1]
+                                    if not found then
+                                        print("Could not find the dependency '" .. dep .. "' in the dependency manifest.")
+                                        os.exit(1)
+                                    end
+                                    print("    * " .. found.name .. "-" .. found.version .. " (" .. found.path .. ", " .. found.version .. ")")
+                                end
+                            end
+                        end
+                        print()
+                    end
+
+                end
+            end
+            return 0
+
+        end
+    },
+
+    -- Selftest of LuaDist.
+    ["selftest"] = {
+        help = [[
+Usage: luadist [TEST_DIRECTORY] selftest [-VARIABLES...]
+
+    The 'selftest' command runs tests of LuaDist, located in TEST_DIRECTORY and
+    displays the results.
+
+    If no TEST_DIRECTORY is specified, the default test directory of LuaDist
+    deployment directory (i.e. ']] .. cfg.test_dir .. [[') is used.
+
+    Optional LuaDist configuration VARIABLES (e.g. -variable=value) can be
+    specified.
+        ]],
+
+        run = function (test_dir)
+            test_dir = test_dir or dist.get_deploy_dir()
+            assert(type(test_dir) == "string", "luadist.selftest: Argument 'deploy_dir' is not a string.")
+            test_dir = sys.abs_path(test_dir)
+
+            -- if the default parameter (i.e. deploy_dir) is passed, use the default test_dir
+            if test_dir == dist.get_deploy_dir() then
+                test_dir = sys.make_path(test_dir, cfg.test_dir)
+            end
+
+            -- try to get an iterator over test files and check it
+            local test_iterator, err = sys.get_directory(test_dir)
+            if not test_iterator then
+                print("Running tests from '" .. test_dir .. "' failed: " .. err)
+                os.exit(1)
+            end
+
+            -- run the tests
+            print("\nRunning tests:")
+            print("==============")
+            for test_file in sys.get_directory(test_dir) do
+                test_file = sys.make_path(test_dir, test_file)
+                if sys.is_file(test_file) then
+                    print()
+                    print(sys.extract_name(test_file) .. ":")
+                    dofile(test_file)
+                end
+            end
+            print()
+            return 0
+        end
+    },
+}
+
+-- Run the functionality of LuaDist 'command' in the 'deploy_dir' with other items
+-- or settings/variables starting at 'other_idx' index of special variable 'arg'.
+local function run_command(deploy_dir, command, other_idx)
+    deploy_dir = deploy_dir or dist.get_deploy_dir()
+    assert(type(deploy_dir) == "string", "luadist.run_command: Argument 'deploy_dir' is not a string.")
+    assert(type(command) == "string", "luadist.run_command: Argument 'command' is not a string.")
+    assert(not other_idx or type(other_idx) == "number", "luadist.run_command: Argument 'other_idx' is not a number.")
+    deploy_dir = sys.abs_path(deploy_dir)
+
+    local items = {}
+    local cmake_variables = {}
+
+    -- parse items after the command (and LuaDist or CMake variables)
+    if other_idx then
+        for i = other_idx, #arg do
+
+            -- CMake variable
+            if arg[i]:match("^%-D(.-)=(.*)$") then
+                local variable, value = arg[i]:match("^%-D(.-)=(.*)$")
+                cmake_variables[variable] = value
+
+            -- LuaDist variable
+            elseif arg[i]:match("^%-(.-)=(.*)$") then
+                local variable, value = arg[i]:match("^%-(.-)=(.*)$")
+                apply_settings(variable, value)
+
+            -- LuaDist boolean variable with implicit 'true' value
+            elseif arg[i]:match("^%-(.-)$") then
+                local variable, value = arg[i]:match("^%-(.-)$")
+                apply_settings(variable, "true")
+
+            -- not a LuaDist or CMake variable
+            else
+                table.insert(items, arg[i])
+            end
+        end
+    end
+
+    -- run the required LuaDist functionality
+    return commands[command].run(sys.abs_path(deploy_dir), items, cmake_variables)
+end
+
+-- Print information about Luadist (version, license, etc.).
+function print_info()
+    print([[
+LuaDist-git ]].. cfg.version .. [[ - Lua package manager for the LuaDist deployment system.
+Released under the MIT License. See https://github.com/luadist/luadist-git
+          ]])
+    return 0
+end
+
+-- Convenience function for printing the main luadist help.
+function print_help()
+    return run_command(nil, "help")
+end
+
+-- Set the LuaDist 'variable' to the 'value'.
+-- See available settings in 'dist.config' module.
+function apply_settings(variable, value)
+    assert(type(variable) == "string", "luadist.apply_settings: Argument 'variable' is not a string.")
+    assert(type(value) == "string", "luadist.apply_settings: Argument 'value' is not a string.")
+
+    -- check whether the settings variable exists
+    if cfg[variable] == nil then
+        print("Unknown LuaDist configuration option: '" .. variable .. "'.")
+        os.exit(1)
+
+    -- ensure the right type
+
+    elseif type(cfg[variable]) == "boolean" then
+        value = value:lower()
+        if value == "true" or value == "yes" or value == "on" or value == "1" then
+            value = true
+        elseif value == "false" or value == "no" or value == "off" or value == "0" then
+            value = false
+        else
+            print("Value of LuaDist option '" .. variable .. "' must be a boolean.")
+            os.exit(1)
+        end
+
+    elseif type(cfg[variable]) == "number" then
+        value = tonumber(value)
+        if not value then
+            print("Value of LuaDist option '" .. variable .. "' must be a number.")
+            os.exit(1)
+        end
+
+    elseif type(cfg[variable]) == "table" then
+        local err
+        value, err = utils.make_table(value, ",")
+        if not value then
+            print("Error when parsing the LuaDist variable '" .. variable .. "': " .. err)
+            os.exit(1)
+        end
+    end
+
+    -- set the LuaDist variable
+    cfg[variable] = value
+
+end
+
+-- Parse command line input and run the required command.
+if pcall(debug.getlocal, 4, 1) then
+    return commands -- return commands when used as module
+elseif not commands[arg[1]] and commands[arg[2]] then
+    -- deploy_dir specified
+    return run_command(arg[1], arg[2], 3)
+elseif commands[arg[1]] then
+    -- deploy_dir not specified
+    return run_command(dist.get_deploy_dir(), arg[1], 2)
+else
+    -- unknown command
+    if arg[1] then
+        print("Unknown command '" .. arg[1] .. "'. Printing help...\n")
+        print_help()
+        os.exit(1)
+    end
+    return print_help()
+end

lualibs/metalua/lopcodes.lua

@@ -281,7 +281,7 @@ luaP.OpCode = {}   -- lookup name -> number
 luaP.ROpCode = {}  -- lookup number -> name
 
 local i = 0
-for v in string.gfind([[
+for v in string.gmatch([[
 MOVE -- 0
 LOADK
 LOADBOOL

lualibs/mime.lua

@@ -2,7 +2,6 @@
 -- MIME support for the Lua language.
 -- Author: Diego Nehab
 -- Conforming to RFCs 2045-2049
--- RCS ID: $Id: mime.lua 1418 2006-04-25 09:38:15Z 3rdparty $
 -----------------------------------------------------------------------------
 
 -----------------------------------------------------------------------------
@@ -13,12 +12,14 @@ local ltn12 = require("ltn12")
 local mime = require("mime.core")
 local io = require("io")
 local string = require("string")
-module("mime")
+local _M = mime
 
 -- encode, decode and wrap algorithm tables
-encodet = {}
-decodet = {}
-wrapt = {}
+local encodet, decodet, wrapt = {},{},{}
+
+_M.encodet = encodet
+_M.decodet = decodet
+_M.wrapt   = wrapt  
 
 -- creates a function that chooses a filter by name from a given table
 local function choose(table)
@@ -27,28 +28,29 @@ local function choose(table)
             name, opt1, opt2 = "default", name, opt1
         end
         local f = table[name or "nil"]
-        if not f then error("unknown key (" .. base.tostring(name) .. ")", 3)
+        if not f then 
+            base.error("unknown key (" .. base.tostring(name) .. ")", 3)
         else return f(opt1, opt2) end
     end
 end
 
 -- define the encoding filters
 encodet['base64'] = function()
-    return ltn12.filter.cycle(b64, "")
+    return ltn12.filter.cycle(_M.b64, "")
 end
 
 encodet['quoted-printable'] = function(mode)
-    return ltn12.filter.cycle(qp, "",
+    return ltn12.filter.cycle(_M.qp, "",
         (mode == "binary") and "=0D=0A" or "\r\n")
 end
 
 -- define the decoding filters
 decodet['base64'] = function()
-    return ltn12.filter.cycle(unb64, "")
+    return ltn12.filter.cycle(_M.unb64, "")
 end
 
 decodet['quoted-printable'] = function()
-    return ltn12.filter.cycle(unqp, "")
+    return ltn12.filter.cycle(_M.unqp, "")
 end
 
 local function format(chunk)
@@ -61,26 +63,28 @@ end
 -- define the line-wrap filters
 wrapt['text'] = function(length)
     length = length or 76
-    return ltn12.filter.cycle(wrp, length, length)
+    return ltn12.filter.cycle(_M.wrp, length, length)
 end
 wrapt['base64'] = wrapt['text']
 wrapt['default'] = wrapt['text']
 
 wrapt['quoted-printable'] = function()
-    return ltn12.filter.cycle(qpwrp, 76, 76)
+    return ltn12.filter.cycle(_M.qpwrp, 76, 76)
 end
 
 -- function that choose the encoding, decoding or wrap algorithm
-encode = choose(encodet)
-decode = choose(decodet)
-wrap = choose(wrapt)
+_M.encode = choose(encodet)
+_M.decode = choose(decodet)
+_M.wrap = choose(wrapt)
 
 -- define the end-of-line normalization filter
-function normalize(marker)
-    return ltn12.filter.cycle(eol, 0, marker)
+function _M.normalize(marker)
+    return ltn12.filter.cycle(_M.eol, 0, marker)
 end
 
 -- high level stuffing filter
-function stuff()
-    return ltn12.filter.cycle(dot, 2)
+function _M.stuff()
+    return ltn12.filter.cycle(_M.dot, 2)
 end
+
+return _M

lualibs/mobdebug/mobdebug.lua

@@ -1,12 +1,12 @@
 --
--- MobDebug 0.5401
+-- MobDebug 0.55
 -- Copyright 2011-13 Paul Kulchenko
 -- Based on RemDebug 1.0 Copyright Kepler Project 2005
 --
 
 local mobdebug = {
   _NAME = "mobdebug",
-  _VERSION = 0.5401,
+  _VERSION = 0.55,
   _COPYRIGHT = "Paul Kulchenko",
   _DESCRIPTION = "Mobile Remote Debugger for the Lua programming language",
   port = os and os.getenv and os.getenv("MOBDEBUG_PORT") or 8172,
@@ -61,11 +61,12 @@ end
 -- check for OS and convert file names to lower case on windows
 -- (its file system is case insensitive, but case preserving), as setting a
 -- breakpoint on x:\Foo.lua will not work if the file was loaded as X:\foo.lua.
--- OSX and Windows behave the same way (case insensitive, but case preserving)
-local iscasepreserving = os and os.getenv and (os.getenv('WINDIR')
-  or (os.getenv('OS') or ''):match('[Ww]indows')
-  or os.getenv('DYLD_LIBRARY_PATH'))
-  or not io.open("/proc")
+-- OSX and Windows behave the same way (case insensitive, but case preserving).
+-- OSX can be configured to be case-sensitive, so check for that. This doesn't
+-- handle the case of different partitions having different case-sensitivity.
+local win = os and os.getenv and (os.getenv('WINDIR') or (os.getenv('OS') or ''):match('[Ww]indows')) and true or false
+local mac = not win and (os and os.getenv and os.getenv('DYLD_LIBRARY_PATH') or not io.open("/proc")) and true or false
+local iscasepreserving = win or (mac and io.open('/library') ~= nil)
 
 -- turn jit off based on Mike Pall's comment in this discussion:
 -- http://www.freelists.org/post/luajit/Debug-hooks-and-JIT,2
@@ -105,7 +106,7 @@ end
 local function q(s) return s:gsub('([%(%)%.%%%+%-%*%?%[%^%$%]])','%%%1') end
 
 local serpent = (function() ---- include Serpent module for serialization
-local n, v = "serpent", 0.24 -- (C) 2012-13 Paul Kulchenko; MIT License
+local n, v = "serpent", 0.25 -- (C) 2012-13 Paul Kulchenko; MIT License
 local c, d = "Paul Kulchenko", "Lua serializer and pretty printer"
 local snum = {[tostring(1/0)]='1/0 --[[math.huge]]',[tostring(-1/0)]='-1/0 --[[-math.huge]]',[tostring(0/0)]='0/0'}
 local badtype = {thread = true, userdata = true, cdata = true}
@@ -118,7 +119,7 @@ for _,g in ipairs({'coroutine', 'debug', 'io', 'math', 'string', 'table', 'os'})
   for k,v in pairs(G[g]) do globals[v] = g..'.'..k end end
 
 local function s(t, opts)
-  local name, indent, fatal = opts.name, opts.indent, opts.fatal
+  local name, indent, fatal, maxnum = opts.name, opts.indent, opts.fatal, opts.maxnum
   local sparse, custom, huge = opts.sparse, opts.custom, not opts.nohuge
   local space, maxl = (opts.compact and '' or ' '), (opts.maxlevel or math.huge)
   local iname, comm = '_'..(name or ''), opts.comment and (tonumber(opts.comment) or math.huge)
@@ -151,7 +152,7 @@ local function s(t, opts)
       ((type(name) == "number") and '' or name..space..'='..space) or
       (name ~= nil and sname..space..'='..space or '')
     if seen[t] then -- already seen this element
-      table.insert(sref, spath..space..'='..space..seen[t])
+      sref[#sref+1] = spath..space..'='..space..seen[t]
       return tag..'nil'..comment('ref', level) end
     if type(mt) == 'table' and (mt.__serialize or mt.__tostring) then -- knows how to serialize itself
       seen[t] = insref or spath
@@ -161,10 +162,14 @@ local function s(t, opts)
       if level >= maxl then return tag..'{}'..comment('max', level) end
       seen[t] = insref or spath
       if next(t) == nil then return tag..'{}'..comment(t, level) end -- table empty
-      local maxn, o, out = #t, {}, {}
-      for key = 1, maxn do table.insert(o, key) end
-      for key in pairs(t) do if not o[key] or key > maxn then table.insert(o, key) end end
-      if opts.sortkeys then alphanumsort(o, t, opts.sortkeys) end
+      local maxn, o, out = math.min(#t, maxnum or #t), {}, {}
+      for key = 1, maxn do o[key] = key end
+      if not maxnum or #o < maxnum then
+        local n = #o -- n = n + 1; o[n] is much faster than o[#o+1] on large tables
+        for key in pairs(t) do if o[key] ~= key then n = n + 1; o[n] = key end end end
+      if maxnum and #o > maxnum then o[maxnum+1] = nil end
+      if opts.sortkeys and #o > maxn then alphanumsort(o, t, opts.sortkeys) end
+      local sparse = sparse and #o > maxn -- disable sparsness if only numeric keys (shorter output)
       for n, key in ipairs(o) do
         local value, ktype, plainindex = t[key], type(key), n <= maxn and not sparse
         if opts.valignore and opts.valignore[value] -- skip ignored values; do nothing
@@ -173,14 +178,14 @@ local function s(t, opts)
         or sparse and value == nil then -- skipping nils; do nothing
         elseif ktype == 'table' or ktype == 'function' or badtype[ktype] then
           if not seen[key] and not globals[key] then
-            table.insert(sref, 'placeholder')
+            sref[#sref+1] = 'placeholder'
             local sname = safename(iname, gensym(key)) -- iname is table for local variables
             sref[#sref] = val2str(key,sname,indent,sname,iname,true) end
-          table.insert(sref, 'placeholder')
+          sref[#sref+1] = 'placeholder'
           local path = seen[t]..'['..(seen[key] or globals[key] or gensym(key))..']'
           sref[#sref] = path..space..'='..space..(seen[value] or val2str(value,nil,indent,path))
         else
-          table.insert(out,val2str(value,key,indent,insref,seen[t],plainindex,level+1))
+          out[#out+1] = val2str(value,key,indent,insref,seen[t],plainindex,level+1)
         end
       end
       local prefix = string.rep(indent or '', level)
@@ -213,6 +218,9 @@ return { _NAME = n, _COPYRIGHT = c, _DESCRIPTION = d, _VERSION = v, serialize =
   block = function(a, opts) return s(a, merge({indent = '  ', sortkeys = true, comment = true}, opts)) end }
 end)() ---- end of Serpent module
 
+mobdebug.line = serpent.line
+mobdebug.dump = serpent.dump
+
 local function removebasedir(path, basedir)
   if iscasepreserving then
     -- check if the lowercased path matches the basedir
@@ -408,9 +416,13 @@ local function debug_hook(event, line)
   -- the next line checks if the debugger is run under LuaJIT and if
   -- one of debugger methods is present in the stack, it simply returns.
   if jit then
-    local coro = coroutine.running()
-    if coro_debugee and coro ~= coro_debugee and not coroutines[coro]
-      or not coro_debugee and (in_debugger() or coro and not coroutines[coro])
+    -- when luajit is compiled with LUAJIT_ENABLE_LUA52COMPAT,
+    -- coroutine.running() returns non-nil for the main thread.
+    local coro, main = coroutine.running()
+    if not coro or main then coro = 'main' end
+    local disabled = coroutines[coro] == false
+      or coroutines[coro] == nil and coro ~= (coro_debugee or 'main')
+    if coro_debugee and disabled or not coro_debugee and (disabled or in_debugger())
     then return end
   end
 
@@ -479,7 +491,7 @@ local function debug_hook(event, line)
         -- this is either a file name coming from loadstring("chunk", "file"),
         -- or the actual source code that needs to be serialized (as it may
         -- include newlines); assume it's a file name if it's all on one line.
-        file = file:find("[\r\n]") and serpent.line(file) or file
+        file = file:find("[\r\n]") and mobdebug.line(file) or file
       end
 
       -- set to true if we got here; this only needs to be done once per
@@ -505,7 +517,9 @@ local function debug_hook(event, line)
     -- no watch that was fired. If there was a watch, handle its result.
     local getin = (status == nil) and
       (step_into
-      or (step_over and stack_level <= step_level)
+      -- when coroutine.running() return `nil` (main thread in Lua 5.1),
+      -- step_over will equal 'main', so need to check for that explicitly.
+      or (step_over and step_over == (coroutine.running() or 'main') and stack_level <= step_level)
       or has_breakpoint(file, line)
       or is_pending(server))
 
@@ -540,6 +554,9 @@ local function debug_hook(event, line)
     end
 
     if vars then restore_vars(vars) end
+
+    -- last command requested Step Over/Out; store the current thread
+    if step_over == true then step_over = coroutine.running() or 'main' end
   end
 end
 
@@ -548,13 +565,13 @@ local function stringify_results(status, ...)
 
   local t = {...}
   for i,v in pairs(t) do -- stringify each of the returned values
-    local ok, res = pcall(serpent.line, v, {nocode = true, comment = 1})
+    local ok, res = pcall(mobdebug.line, v, {nocode = true, comment = 1})
     t[i] = ok and res or ("%q"):format(res):gsub("\010","n"):gsub("\026","\\026")
   end
   -- stringify table with all returned values
   -- this is done to allow each returned value to be used (serialized or not)
   -- intependently and to preserve "original" comments
-  return pcall(serpent.dump, t, {sparse = false})
+  return pcall(mobdebug.dump, t, {sparse = false})
 end
 
 local function debugger_loop(sev, svars, sfile, sline)
@@ -780,7 +797,7 @@ local function debugger_loop(sev, svars, sfile, sline)
         server:send("401 Error in Execution " .. #vars .. "\n")
         server:send(vars)
       else
-        local ok, res = pcall(serpent.dump, vars, {nocode = true, sparse = false})
+        local ok, res = pcall(mobdebug.dump, vars, {nocode = true, sparse = false})
         if ok then
           server:send("200 OK " .. res .. "\n")
         else
@@ -792,19 +809,22 @@ local function debugger_loop(sev, svars, sfile, sline)
       local _, _, stream, mode = string.find(line, "^[A-Z]+%s+(%w+)%s+([dcr])%s*$")
       if stream and mode and stream == "stdout" then
         -- assign "print" in the global environment
-        genv.print = mode == 'd' and iobase.print or coroutine.wrap(function(...)
+        local default = mode == 'd'
+        genv.print = default and iobase.print or coroutine.wrap(function()
           -- wrapping into coroutine.wrap protects this function from
-          -- being stepped through in the debugger
-          local tbl = {...}
+          -- being stepped through in the debugger.
+          -- don't use vararg (...) as it adds a reference for its values,
+          -- which may affect how they are garbage collected
           while true do
+            local tbl = {coroutine.yield()}
             if mode == 'c' then iobase.print(unpack(tbl)) end
             for n = 1, #tbl do
-              tbl[n] = select(2, pcall(serpent.line, tbl[n], {nocode = true, comment = false})) end
+              tbl[n] = select(2, pcall(mobdebug.line, tbl[n], {nocode = true, comment = false})) end
             local file = table.concat(tbl, "\t").."\n"
             server:send("204 Output " .. stream .. " " .. #file .. "\n" .. file)
-            tbl = {coroutine.yield()}
           end
         end)
+        if not default then genv.print() end -- "fake" print to start printing loop
         server:send("200 OK\n")
       else
         server:send("400 Bad Request\n")
@@ -839,7 +859,8 @@ local function start(controller_host, controller_port)
   controller_host = lasthost or "localhost"
   controller_port = lastport or mobdebug.port
 
-  server = (socket.connect4 or socket.connect)(controller_host, controller_port)
+  local err
+  server, err = (socket.connect4 or socket.connect)(controller_host, controller_port)
   if server then
     -- correct stack depth which already has some calls on it
     -- so it doesn't go into negative when those calls return
@@ -876,7 +897,8 @@ local function start(controller_host, controller_port)
     step_into = true -- start with step command
     return true
   else
-    print("Could not connect to " .. controller_host .. ":" .. controller_port)
+    print(("Could not connect to %s:%s: %s")
+      :format(controller_host, controller_port, err or "unknown error"))
   end
 end
 
@@ -891,7 +913,8 @@ local function controller(controller_host, controller_port, scratchpad)
   controller_port = lastport or mobdebug.port
 
   local exitonerror = not scratchpad
-  server = (socket.connect4 or socket.connect)(controller_host, controller_port)
+  local err
+  server, err = (socket.connect4 or socket.connect)(controller_host, controller_port)
   if server then
     local function report(trace, err)
       local msg = err .. "\n" .. trace
@@ -938,7 +961,8 @@ local function controller(controller_host, controller_port, scratchpad)
       end
     end
   else
-    print("Could not connect to " .. controller_host .. ":" .. controller_port)
+    print(("Could not connect to %s:%s: %s")
+      :format(controller_host, controller_port, err or "unknown error"))
     return false
   end
   return true
@@ -955,13 +979,15 @@ end
 local function on()
   if not (isrunning() and server) then return end
 
-  local co = coroutine.running()
+  -- main is set to true under Lua5.2 for the "main" chunk.
+  -- Lua5.1 returns co as `nil` in that case.
+  local co, main = coroutine.running()
+  if main then co = nil end
   if co then
-    if not coroutines[co] then
-      coroutines[co] = true
-      debug.sethook(co, debug_hook, "lcr")
-    end
+    coroutines[co] = true
+    debug.sethook(co, debug_hook, "lcr")
   else
+    if jit then coroutines.main = true end
     debug.sethook(debug_hook, "lcr")
   end
 end
@@ -969,13 +995,28 @@ end
 local function off()
   if not (isrunning() and server) then return end
 
-  local co = coroutine.running()
+  -- main is set to true under Lua5.2 for the "main" chunk.
+  -- Lua5.1 returns co as `nil` in that case.
+  local co, main = coroutine.running()
+  if main then co = nil end
+
+  -- don't remove coroutine hook under LuaJIT as there is only one (global) hook
   if co then
-    if coroutines[co] then coroutines[co] = false end
-    -- don't remove coroutine hook under LuaJIT as there is only one (global) hook
+    coroutines[co] = false
     if not jit then debug.sethook(co) end
   else
-    debug.sethook()
+    if jit then coroutines.main = false end
+    if not jit then debug.sethook() end
+  end
+
+  -- check if there is any thread that is still being debugged under LuaJIT;
+  -- if not, turn the debugging off
+  if jit then
+    local remove = true
+    for co, debugged in pairs(coroutines) do
+      if debugged then remove = false; break end
+    end
+    if remove then debug.sethook() end
   end
 end
 
@@ -1146,9 +1187,10 @@ local function handle(params, client, options)
           -- if file is not open and winapi is there, try with a short path;
           -- this may be needed for unicode paths on windows
           winapi.set_encoding(winapi.CP_UTF8)
-          file = io.open(winapi.short_path(exp), "r")
+          local shortp = winapi.short_path(exp)
+          file = shortp and io.open(shortp, "r")
         end
-        if not file then error("Cannot open file " .. exp) end
+        if not file then return nil, nil, "Cannot open file " .. exp end
         -- read the file and remove the shebang line as it causes a compilation error
         local lines = file:read("*all"):gsub("^#!.-\n", "\n")
         file:close()
@@ -1242,7 +1284,7 @@ local function handle(params, client, options)
         return nil, nil, stack
       end
       for _,frame in ipairs(stack) do
-        print(serpent.line(frame[1], {comment = false}))
+        print(mobdebug.line(frame[1], {comment = false}))
       end
       return stack
     elseif status == "401" then
@@ -1379,7 +1421,9 @@ local function moai()
   if not moconew then return end
   MOAICoroutine.new = function(...)
     local thread = moconew(...)
-    local mt = getmetatable(thread)
+    -- need to support both thread.run and getmetatable(thread).run, which
+    -- was used in earlier MOAI versions
+    local mt = thread.run and thread or getmetatable(thread)
     local patched = mt.run
     mt.run = function(self, f, ...)
       return patched(self,  function(...)
@@ -1412,6 +1456,8 @@ local function done()
 end
 
 -- make public functions available
+mobdebug.setbreakpoint = set_breakpoint
+mobdebug.removebreakpoint = remove_breakpoint
 mobdebug.listen = listen
 mobdebug.loop = loop
 mobdebug.scratchpad = scratchpad
@@ -1423,8 +1469,6 @@ mobdebug.off = off
 mobdebug.moai = moai
 mobdebug.coro = coro
 mobdebug.done = done
-mobdebug.line = serpent.line
-mobdebug.dump = serpent.dump
 mobdebug.yield = nil -- callback
 
 -- this is needed to make "require 'modebug'" to work when mobdebug

lualibs/socket.lua

@@ -1,7 +1,6 @@
 -----------------------------------------------------------------------------
 -- LuaSocket helper module
 -- Author: Diego Nehab
--- RCS ID: $Id: socket.lua 1418 2006-04-25 09:38:15Z 3rdparty $
 -----------------------------------------------------------------------------
 
 -----------------------------------------------------------------------------
@@ -11,12 +10,13 @@ local base = _G
 local string = require("string")
 local math = require("math")
 local socket = require("socket.core")
-module("socket")
 
------------------------------------------------------------------------------
--- Exported auxiliar functions
------------------------------------------------------------------------------
-function connect(address, port, laddress, lport)
+local _M = socket
+
+-- this is needed in case this library is used when "socket.core" is loaded,
+-- but has an older version of luasocket that does not include `connect`.
+if not socket.connect then
+  socket.connect = function (address, port, laddress, lport)
     local sock, err = socket.tcp()
     if not sock then return nil, err end
     if laddress then
@@ -26,22 +26,52 @@ function connect(address, port, laddress, lport)
     local res, err = sock:connect(address, port)
     if not res then return nil, err end
     return sock
+  end
 end
 
-function bind(host, port, backlog)
-    local sock, err = socket.tcp()
-    if not sock then return nil, err end
-    sock:setoption("reuseaddr", true)
-    local res, err = sock:bind(host, port)
-    if not res then return nil, err end
-    res, err = sock:listen(backlog)
-    if not res then return nil, err end
-    return sock
+-----------------------------------------------------------------------------
+-- Exported auxiliar functions
+-----------------------------------------------------------------------------
+function _M.connect4(address, port, laddress, lport)
+    return socket.connect(address, port, laddress, lport, "inet")
 end
 
-try = newtry()
+function _M.connect6(address, port, laddress, lport)
+    return socket.connect(address, port, laddress, lport, "inet6")
+end
 
-function choose(table)
+function _M.bind(host, port, backlog)
+    if host == "*" then host = "0.0.0.0" end
+    local addrinfo, err = socket.dns.getaddrinfo(host);
+    if not addrinfo then return nil, err end
+    local sock, res
+    err = "no info on address"
+    for i, alt in base.ipairs(addrinfo) do
+        if alt.family == "inet" then
+            sock, err = socket.tcp()
+        else
+            sock, err = socket.tcp6()
+        end
+        if not sock then return nil, err end
+        sock:setoption("reuseaddr", true)
+        res, err = sock:bind(alt.addr, port)
+        if not res then 
+            sock:close()
+        else 
+            res, err = sock:listen(backlog)
+            if not res then 
+                sock:close()
+            else
+                return sock
+            end
+        end 
+    end
+    return nil, err
+end
+
+_M.try = _M.newtry()
+
+function _M.choose(table)
     return function(name, opt1, opt2)
         if base.type(name) ~= "string" then
             name, opt1, opt2 = "default", name, opt1
@@ -56,10 +86,11 @@ end
 -- Socket sources and sinks, conforming to LTN12
 -----------------------------------------------------------------------------
 -- create namespaces inside LuaSocket namespace
-sourcet = {}
-sinkt = {}
+local sourcet, sinkt = {}, {}
+_M.sourcet = sourcet
+_M.sinkt = sinkt
 
-BLOCKSIZE = 2048
+_M.BLOCKSIZE = 2048
 
 sinkt["close-when-done"] = function(sock)
     return base.setmetatable({
@@ -89,7 +120,7 @@ end
 
 sinkt["default"] = sinkt["keep-open"]
 
-sink = choose(sinkt)
+_M.sink = _M.choose(sinkt)
 
 sourcet["by-length"] = function(sock, length)
     return base.setmetatable({
@@ -129,5 +160,6 @@ end
 
 sourcet["default"] = sourcet["until-closed"]
 
-source = choose(sourcet)
+_M.source = _M.choose(sourcet)
 
+return _M

lualibs/socket/ftp.lua

@@ -2,7 +2,6 @@
 -- FTP support for the Lua language
 -- LuaSocket toolkit.
 -- Author: Diego Nehab
--- RCS ID: $Id: ftp.lua 1418 2006-04-25 09:38:15Z 3rdparty $
 -----------------------------------------------------------------------------
 
 -----------------------------------------------------------------------------
@@ -16,27 +15,27 @@ local socket = require("socket")
 local url = require("socket.url")
 local tp = require("socket.tp")
 local ltn12 = require("ltn12")
-module("socket.ftp")
-
+socket.ftp = {}
+local _M = socket.ftp
 -----------------------------------------------------------------------------
 -- Program constants
 -----------------------------------------------------------------------------
 -- timeout in seconds before the program gives up on a connection
-TIMEOUT = 60
+_M.TIMEOUT = 60
 -- default port for ftp service
-PORT = 21
+_M.PORT = 21
 -- this is the default anonymous password. used when no password is
 -- provided in url. should be changed to your e-mail.
-USER = "ftp"
-PASSWORD = "anonymous@anonymous.org"
+_M.USER = "ftp"
+_M.PASSWORD = "anonymous@anonymous.org"
 
 -----------------------------------------------------------------------------
 -- Low level FTP API
 -----------------------------------------------------------------------------
 local metat = { __index = {} }
 
-function open(server, port, create)
-    local tp = socket.try(tp.connect(server, port or PORT, create, TIMEOUT))
+function _M.open(server, port, create)
+    local tp = socket.try(tp.connect(server, port or _M.PORT, _M.TIMEOUT, create))
     local f = base.setmetatable({ tp = tp }, metat)
     -- make sure everything gets closed in an exception
     f.try = socket.newtry(function() f:close() end)
@@ -44,22 +43,22 @@ function open(server, port, create)
 end
 
 function metat.__index:portconnect()
-    self.try(self.server:settimeout(TIMEOUT))
+    self.try(self.server:settimeout(_M.TIMEOUT))
     self.data = self.try(self.server:accept())
-    self.try(self.data:settimeout(TIMEOUT))
+    self.try(self.data:settimeout(_M.TIMEOUT))
 end
 
 function metat.__index:pasvconnect()
     self.data = self.try(socket.tcp())
-    self.try(self.data:settimeout(TIMEOUT))
+    self.try(self.data:settimeout(_M.TIMEOUT))
     self.try(self.data:connect(self.pasvt.ip, self.pasvt.port))
 end
 
 function metat.__index:login(user, password)
-    self.try(self.tp:command("user", user or USER))
+    self.try(self.tp:command("user", user or _M.USER))
     local code, reply = self.try(self.tp:check{"2..", 331})
     if code == 331 then
-        self.try(self.tp:command("pass", password or PASSWORD))
+        self.try(self.tp:command("pass", password or _M.PASSWORD))
         self.try(self.tp:check("2.."))
     end
     return 1
@@ -88,7 +87,7 @@ function metat.__index:port(ip, port)
         ip, port = self.try(self.tp:getcontrol():getsockname())
         self.server = self.try(socket.bind(ip, 0))
         ip, port = self.try(self.server:getsockname())
-        self.try(server:settimeout(TIMEOUT))
+        self.try(self.server:settimeout(_M.TIMEOUT))
     end
     local pl = math.mod(port, 256)
     local ph = (port - pl)/256
@@ -116,10 +115,11 @@ function metat.__index:send(sendt)
     if not self.pasvt then self:portconnect() end
     -- get the sink, source and step for the transfer
     local step = sendt.step or ltn12.pump.step
+    local readt = {self.tp.c}
     local checkstep = function(src, snk)
         -- check status in control connection while downloading
         local readyt = socket.select(readt, nil, 0)
-        if readyt[tp] then self.try(self.tp:check("2..")) end
+        if readyt[tp] then code = self.try(self.tp:check("2..")) end
         return step(src, snk)
     end
     local sink = socket.sink("close-when-done", self.data)
@@ -142,7 +142,11 @@ function metat.__index:receive(recvt)
     if argument == "" then argument = nil end
     local command = recvt.command or "retr"
     self.try(self.tp:command(command, argument))
-    local code = self.try(self.tp:check{"1..", "2.."})
+    local code,reply = self.try(self.tp:check{"1..", "2.."})
+    if (code >= 200) and (code <= 299) then
+        recvt.sink(reply)
+        return 1
+    end
     if not self.pasvt then self:portconnect() end
     local source = socket.source("until-closed", self.data)
     local step = recvt.step or ltn12.pump.step
@@ -186,9 +190,9 @@ end
 -----------------------------------------------------------------------------
 -- High level FTP API
 -----------------------------------------------------------------------------
-function override(t)
+local function override(t)
     if t.url then
-        u = url.parse(t.url)
+        local u = url.parse(t.url)
         for i,v in base.pairs(t) do
             u[i] = v
         end
@@ -199,7 +203,7 @@ end
 local function tput(putt)
     putt = override(putt)
     socket.try(putt.host, "missing hostname")
-    local f = open(putt.host, putt.port, putt.create)
+    local f = _M.open(putt.host, putt.port, putt.create)
     f:greet()
     f:login(putt.user, putt.password)
     if putt.type then f:type(putt.type) end
@@ -211,8 +215,8 @@ local function tput(putt)
 end
 
 local default = {
-	path = "/",
-	scheme = "ftp"
+    path = "/",
+    scheme = "ftp"
 }
 
 local function parse(u)
@@ -234,7 +238,7 @@ local function sput(u, body)
     return tput(putt)
 end
 
-put = socket.protect(function(putt, body)
+_M.put = socket.protect(function(putt, body)
     if base.type(putt) == "string" then return sput(putt, body)
     else return tput(putt) end
 end)
@@ -242,7 +246,7 @@ end)
 local function tget(gett)
     gett = override(gett)
     socket.try(gett.host, "missing hostname")
-    local f = open(gett.host, gett.port, gett.create)
+    local f = _M.open(gett.host, gett.port, gett.create)
     f:greet()
     f:login(gett.user, gett.password)
     if gett.type then f:type(gett.type) end
@@ -260,7 +264,7 @@ local function sget(u)
     return table.concat(t)
 end
 
-command = socket.protect(function(cmdt)
+_M.command = socket.protect(function(cmdt)
     cmdt = override(cmdt)
     socket.try(cmdt.host, "missing hostname")
     socket.try(cmdt.command, "missing command")
@@ -273,8 +277,9 @@ command = socket.protect(function(cmdt)
     return f:close()
 end)
 
-get = socket.protect(function(gett)
+_M.get = socket.protect(function(gett)
     if base.type(gett) == "string" then return sget(gett)
     else return tget(gett) end
 end)
 
+return _M

lualibs/socket/headers.lua

@@ -0,0 +1,104 @@
+-----------------------------------------------------------------------------
+-- Canonic header field capitalization
+-- LuaSocket toolkit.
+-- Author: Diego Nehab
+-----------------------------------------------------------------------------
+local socket = require("socket")
+socket.headers = {}
+local _M = socket.headers
+
+_M.canonic = {
+    ["accept"] = "Accept",
+    ["accept-charset"] = "Accept-Charset",
+    ["accept-encoding"] = "Accept-Encoding",
+    ["accept-language"] = "Accept-Language",
+    ["accept-ranges"] = "Accept-Ranges",
+    ["action"] = "Action",
+    ["alternate-recipient"] = "Alternate-Recipient",
+    ["age"] = "Age",
+    ["allow"] = "Allow",
+    ["arrival-date"] = "Arrival-Date",
+    ["authorization"] = "Authorization",
+    ["bcc"] = "Bcc",
+    ["cache-control"] = "Cache-Control",
+    ["cc"] = "Cc",
+    ["comments"] = "Comments",
+    ["connection"] = "Connection",
+    ["content-description"] = "Content-Description",
+    ["content-disposition"] = "Content-Disposition",
+    ["content-encoding"] = "Content-Encoding",
+    ["content-id"] = "Content-ID",
+    ["content-language"] = "Content-Language",
+    ["content-length"] = "Content-Length",
+    ["content-location"] = "Content-Location",
+    ["content-md5"] = "Content-MD5",
+    ["content-range"] = "Content-Range",
+    ["content-transfer-encoding"] = "Content-Transfer-Encoding",
+    ["content-type"] = "Content-Type",
+    ["cookie"] = "Cookie",
+    ["date"] = "Date",
+    ["diagnostic-code"] = "Diagnostic-Code",
+    ["dsn-gateway"] = "DSN-Gateway",
+    ["etag"] = "ETag",
+    ["expect"] = "Expect",
+    ["expires"] = "Expires",
+    ["final-log-id"] = "Final-Log-ID",
+    ["final-recipient"] = "Final-Recipient",
+    ["from"] = "From",
+    ["host"] = "Host",
+    ["if-match"] = "If-Match",
+    ["if-modified-since"] = "If-Modified-Since",
+    ["if-none-match"] = "If-None-Match",
+    ["if-range"] = "If-Range",
+    ["if-unmodified-since"] = "If-Unmodified-Since",
+    ["in-reply-to"] = "In-Reply-To",
+    ["keywords"] = "Keywords",
+    ["last-attempt-date"] = "Last-Attempt-Date",
+    ["last-modified"] = "Last-Modified",
+    ["location"] = "Location",
+    ["max-forwards"] = "Max-Forwards",
+    ["message-id"] = "Message-ID",
+    ["mime-version"] = "MIME-Version",
+    ["original-envelope-id"] = "Original-Envelope-ID",
+    ["original-recipient"] = "Original-Recipient",
+    ["pragma"] = "Pragma",
+    ["proxy-authenticate"] = "Proxy-Authenticate",
+    ["proxy-authorization"] = "Proxy-Authorization",
+    ["range"] = "Range",
+    ["received"] = "Received",
+    ["received-from-mta"] = "Received-From-MTA",
+    ["references"] = "References",
+    ["referer"] = "Referer",
+    ["remote-mta"] = "Remote-MTA",
+    ["reply-to"] = "Reply-To",
+    ["reporting-mta"] = "Reporting-MTA",
+    ["resent-bcc"] = "Resent-Bcc",
+    ["resent-cc"] = "Resent-Cc",
+    ["resent-date"] = "Resent-Date",
+    ["resent-from"] = "Resent-From",
+    ["resent-message-id"] = "Resent-Message-ID",
+    ["resent-reply-to"] = "Resent-Reply-To",
+    ["resent-sender"] = "Resent-Sender",
+    ["resent-to"] = "Resent-To",
+    ["retry-after"] = "Retry-After",
+    ["return-path"] = "Return-Path",
+    ["sender"] = "Sender",
+    ["server"] = "Server",
+    ["smtp-remote-recipient"] = "SMTP-Remote-Recipient",
+    ["status"] = "Status",
+    ["subject"] = "Subject",
+    ["te"] = "TE",
+    ["to"] = "To",
+    ["trailer"] = "Trailer",
+    ["transfer-encoding"] = "Transfer-Encoding",
+    ["upgrade"] = "Upgrade",
+    ["user-agent"] = "User-Agent",
+    ["vary"] = "Vary",
+    ["via"] = "Via",
+    ["warning"] = "Warning",
+    ["will-retry-until"] = "Will-Retry-Until",
+    ["www-authenticate"] = "WWW-Authenticate",
+    ["x-mailer"] = "X-Mailer",
+}
+
+return _M

lualibs/socket/http.lua

@@ -2,7 +2,6 @@
 -- HTTP/1.1 client support for the Lua language.
 -- LuaSocket toolkit.
 -- Author: Diego Nehab
--- RCS ID: $Id: http.lua 1418 2006-04-25 09:38:15Z 3rdparty $
 -----------------------------------------------------------------------------
 
 -----------------------------------------------------------------------------
@@ -13,9 +12,11 @@ local url = require("socket.url")
 local ltn12 = require("ltn12")
 local mime = require("mime")
 local string = require("string")
+local headers = require("socket.headers")
 local base = _G
 local table = require("table")
-module("socket.http")
+socket.http = {}
+local _M = socket.http
 
 -----------------------------------------------------------------------------
 -- Program constants
@@ -23,9 +24,9 @@ module("socket.http")
 -- connection timeout in seconds
 TIMEOUT = 60
 -- default port for document retrieval
-PORT = 80
+_M.PORT = 80
 -- user agent field sent in request
-USERAGENT = socket._VERSION
+_M.USERAGENT = socket._VERSION
 
 -----------------------------------------------------------------------------
 -- Reads MIME headers from a connection, unfolding where needed
@@ -105,15 +106,15 @@ end
 -----------------------------------------------------------------------------
 local metat = { __index = {} }
 
-function open(host, port, create)
+function _M.open(host, port, create)
     -- create socket with user connect function, or with default
-    local c = socket.try(create or socket.tcp)()
+    local c = socket.try((create or socket.tcp)())
     local h = base.setmetatable({ c = c }, metat)
     -- create finalized try
     h.try = socket.newtry(function() h:close() end)
     -- set timeout before connecting
-    h.try(c:settimeout(TIMEOUT))
-    h.try(c:connect(host, port or PORT))
+    h.try(c:settimeout(_M.TIMEOUT))
+    h.try(c:connect(host, port or _M.PORT))
     -- here everything worked
     return h
 end
@@ -123,10 +124,11 @@ function metat.__index:sendrequestline(method, uri)
     return self.try(self.c:send(reqline))
 end
 
-function metat.__index:sendheaders(headers)
+function metat.__index:sendheaders(tosend)
+    local canonic = headers.canonic
     local h = "\r\n"
-    for i, v in base.pairs(headers) do
-        h = i .. ": " .. v .. "\r\n" .. h
+    for f, v in base.pairs(tosend) do
+        h = (canonic[f] or f) .. ": " .. v .. "\r\n" .. h
     end
     self.try(self.c:send(h))
     return 1
@@ -142,7 +144,12 @@ function metat.__index:sendbody(headers, source, step)
 end
 
 function metat.__index:receivestatusline()
-    local status = self.try(self.c:receive())
+    local status = self.try(self.c:receive(5))
+    -- identify HTTP/0.9 responses, which do not contain a status line
+    -- this is just a heuristic, but is what the RFC recommends
+    if status ~= "HTTP/" then return nil, status end
+    -- otherwise proceed reading a status line
+    status = self.try(self.c:receive("*l", status))
     local code = socket.skip(2, string.find(status, "HTTP/%d*%.%d* (%d%d%d)"))
     return self.try(base.tonumber(code), status)
 end
@@ -163,6 +170,12 @@ function metat.__index:receivebody(headers, sink, step)
         sink, step))
 end
 
+function metat.__index:receive09body(status, sink, step)
+    local source = ltn12.source.rewind(socket.source("until-closed", self.c))
+    source(status)
+    return self.try(ltn12.pump.all(source, sink, step))
+end
+
 function metat.__index:close()
     return self.c:close()
 end
@@ -194,16 +207,21 @@ local function adjustproxy(reqt)
     end
 end
 
-local function adjustheaders(headers, host)
+local function adjustheaders(reqt)
     -- default headers
     local lower = {
-        ["user-agent"] = USERAGENT,
-        ["host"] = host,
+        ["user-agent"] = _M.USERAGENT,
+        ["host"] = reqt.host,
         ["connection"] = "close, TE",
         ["te"] = "trailers"
     }
+    -- if we have authentication information, pass it along
+    if reqt.user and reqt.password then
+        lower["authorization"] = 
+            "Basic " ..  (mime.b64(reqt.user .. ":" .. reqt.password))
+    end
     -- override with user headers
-    for i,v in base.pairs(headers or lower) do
+    for i,v in base.pairs(reqt.headers or lower) do
         lower[string.lower(i)] = v
     end
     return lower
@@ -212,7 +230,7 @@ end
 -- default url parts
 local default = {
     host = "",
-    port = PORT,
+    port = _M.PORT,
     path ="/",
     scheme = "http"
 }
@@ -220,16 +238,17 @@ local default = {
 local function adjustrequest(reqt)
     -- parse url if provided
     local nreqt = reqt.url and url.parse(reqt.url, default) or {}
-    local t = url.parse(reqt.url, default)
     -- explicit components override url
     for i,v in base.pairs(reqt) do nreqt[i] = v end
-    socket.try(nreqt.host, "invalid host '" .. base.tostring(nreqt.host) .. "'")
+    if nreqt.port == "" then nreqt.port = 80 end
+    socket.try(nreqt.host and nreqt.host ~= "", 
+        "invalid host '" .. base.tostring(nreqt.host) .. "'")
     -- compute uri if user hasn't overriden
     nreqt.uri = reqt.uri or adjusturi(nreqt)
     -- ajust host and port if there is a proxy
     nreqt.host, nreqt.port = adjustproxy(nreqt)
     -- adjust headers in request
-    nreqt.headers = adjustheaders(nreqt.headers, nreqt.host)
+    nreqt.headers = adjustheaders(nreqt)
     return nreqt
 end
 
@@ -237,19 +256,11 @@ local function shouldredirect(reqt, code, headers)
     return headers.location and
            string.gsub(headers.location, "%s", "") ~= "" and
            (reqt.redirect ~= false) and
-           (code == 301 or code == 302) and
+           (code == 301 or code == 302 or code == 303 or code == 307) and
            (not reqt.method or reqt.method == "GET" or reqt.method == "HEAD")
            and (not reqt.nredirects or reqt.nredirects < 5)
 end
 
-local function shouldauthorize(reqt, code)
-    -- if there has been an authorization attempt, it must have failed
-    if reqt.headers and reqt.headers["authorization"] then return nil end
-    -- if last attempt didn't fail due to lack of authentication,
-    -- or we don't have authorization information, we can't retry
-    return code == 401 and reqt.user and reqt.password
-end
-
 local function shouldreceivebody(reqt, code)
     if reqt.method == "HEAD" then return nil end
     if code == 204 or code == 304 then return nil end
@@ -258,69 +269,86 @@ local function shouldreceivebody(reqt, code)
 end
 
 -- forward declarations
-local trequest, tauthorize, tredirect
+local trequest, tredirect
 
-function tauthorize(reqt)
-    local auth = "Basic " ..  (mime.b64(reqt.user .. ":" .. reqt.password))
-    reqt.headers["authorization"] = auth
-    return trequest(reqt)
-end
-
-function tredirect(reqt, location)
+--[[local]] function tredirect(reqt, location)
     local result, code, headers, status = trequest {
         -- the RFC says the redirect URL has to be absolute, but some
         -- servers do not respect that
-        url = url.absolute(reqt, location),
+        url = url.absolute(reqt.url, location),
         source = reqt.source,
         sink = reqt.sink,
         headers = reqt.headers,
-        proxy = reqt.proxy,
+        proxy = reqt.proxy, 
         nredirects = (reqt.nredirects or 0) + 1,
-        connect = reqt.connect
-    }
+        create = reqt.create
+    }   
     -- pass location header back as a hint we redirected
+    headers = headers or {}
     headers.location = headers.location or location
     return result, code, headers, status
 end
 
-function trequest(reqt)
-    reqt = adjustrequest(reqt)
-    local h = open(reqt.host, reqt.port, reqt.create)
-    h:sendrequestline(reqt.method, reqt.uri)
-    h:sendheaders(reqt.headers)
-    if reqt.source then h:sendbody(reqt.headers, reqt.source, reqt.step) end
-    local code, headers, status
-    code, status = h:receivestatusline()
+--[[local]] function trequest(reqt)
+    -- we loop until we get what we want, or
+    -- until we are sure there is no way to get it
+    local nreqt = adjustrequest(reqt)
+    local h = _M.open(nreqt.host, nreqt.port, nreqt.create)
+    -- send request line and headers
+    h:sendrequestline(nreqt.method, nreqt.uri)
+    h:sendheaders(nreqt.headers)
+    -- if there is a body, send it
+    if nreqt.source then
+        h:sendbody(nreqt.headers, nreqt.source, nreqt.step) 
+    end
+    local code, status = h:receivestatusline()
+    -- if it is an HTTP/0.9 server, simply get the body and we are done
+    if not code then
+        h:receive09body(status, nreqt.sink, nreqt.step)
+        return 1, 200
+    end
+    local headers
+    -- ignore any 100-continue messages
+    while code == 100 do 
+        headers = h:receiveheaders()
+        code, status = h:receivestatusline()
+    end
     headers = h:receiveheaders()
-    if shouldredirect(reqt, code, headers) then
+    -- at this point we should have a honest reply from the server
+    -- we can't redirect if we already used the source, so we report the error 
+    if shouldredirect(nreqt, code, headers) and not nreqt.source then
         h:close()
         return tredirect(reqt, headers.location)
-    elseif shouldauthorize(reqt, code) then
-        h:close()
-        return tauthorize(reqt)
-    elseif shouldreceivebody(reqt, code) then
-        h:receivebody(headers, reqt.sink, reqt.step)
+    end
+    -- here we are finally done
+    if shouldreceivebody(nreqt, code) then
+        h:receivebody(headers, nreqt.sink, nreqt.step)
     end
     h:close()
     return 1, code, headers, status
 end
 
-local function srequest(u, body)
+local function srequest(u, b)
     local t = {}
     local reqt = {
         url = u,
         sink = ltn12.sink.table(t)
     }
-    if body then
-        reqt.source = ltn12.source.string(body)
-        reqt.headers = { ["content-length"] = string.len(body) }
+    if b then
+        reqt.source = ltn12.source.string(b)
+        reqt.headers = {
+            ["content-length"] = string.len(b),
+            ["content-type"] = "application/x-www-form-urlencoded"
+        }
         reqt.method = "POST"
     end
     local code, headers, status = socket.skip(1, trequest(reqt))
     return table.concat(t), code, headers, status
 end
 
-request = socket.protect(function(reqt, body)
+_M.request = socket.protect(function(reqt, body)
     if base.type(reqt) == "string" then return srequest(reqt, body)
     else return trequest(reqt) end
 end)
+
+return _M

lualibs/socket/smtp.lua

@@ -2,7 +2,6 @@
 -- SMTP client support for the Lua language.
 -- LuaSocket toolkit.
 -- Author: Diego Nehab
--- RCS ID: $Id: smtp.lua 1418 2006-04-25 09:38:15Z 3rdparty $
 -----------------------------------------------------------------------------
 
 -----------------------------------------------------------------------------
@@ -16,23 +15,26 @@ local os = require("os")
 local socket = require("socket")
 local tp = require("socket.tp")
 local ltn12 = require("ltn12")
+local headers = require("socket.headers")
 local mime = require("mime")
-module("socket.smtp")
+
+socket.smtp = {}
+local _M = socket.smtp
 
 -----------------------------------------------------------------------------
 -- Program constants
 -----------------------------------------------------------------------------
 -- timeout for connection
-TIMEOUT = 60
+_M.TIMEOUT = 60
 -- default server used to send e-mails
-SERVER = "localhost"
+_M.SERVER = "localhost"
 -- default port
-PORT = 25
+_M.PORT = 25
 -- domain used in HELO command and default sendmail
 -- If we are under a CGI, try to get from environment
-DOMAIN = os.getenv("SERVER_NAME") or "localhost"
+_M.DOMAIN = os.getenv("SERVER_NAME") or "localhost"
 -- default time zone (means we don't know)
-ZONE = "-0000"
+_M.ZONE = "-0000"
 
 ---------------------------------------------------------------------------
 -- Low level SMTP API
@@ -41,7 +43,7 @@ local metat = { __index = {} }
 
 function metat.__index:greet(domain)
     self.try(self.tp:check("2.."))
-    self.try(self.tp:command("EHLO", domain or DOMAIN))
+    self.try(self.tp:command("EHLO", domain or _M.DOMAIN))
     return socket.skip(1, self.try(self.tp:check("2..")))
 end
 
@@ -75,9 +77,9 @@ end
 function metat.__index:login(user, password)
     self.try(self.tp:command("AUTH", "LOGIN"))
     self.try(self.tp:check("3.."))
-    self.try(self.tp:command(mime.b64(user)))
+    self.try(self.tp:send(mime.b64(user) .. "\r\n"))
     self.try(self.tp:check("3.."))
-    self.try(self.tp:command(mime.b64(password)))
+    self.try(self.tp:send(mime.b64(password) .. "\r\n"))
     return self.try(self.tp:check("2.."))
 end
 
@@ -111,9 +113,9 @@ function metat.__index:send(mailt)
     self:data(ltn12.source.chain(mailt.source, mime.stuff()), mailt.step)
 end
 
-function open(server, port, create)
-    local tp = socket.try(tp.connect(server or SERVER, port or PORT,
-        create, TIMEOUT))
+function _M.open(server, port, create)
+    local tp = socket.try(tp.connect(server or _M.SERVER, port or _M.PORT,
+        _M.TIMEOUT, create))
     local s = base.setmetatable({tp = tp}, metat)
     -- make sure tp is closed if we get an exception
     s.try = socket.newtry(function()
@@ -122,6 +124,15 @@ function open(server, port, create)
     return s
 end
 
+-- convert headers to lowercase
+local function lower_headers(headers)
+    local lower = {}
+    for i,v in base.pairs(headers or lower) do
+        lower[string.lower(i)] = v
+    end
+    return lower
+end
+
 ---------------------------------------------------------------------------
 -- Multipart message source
 -----------------------------------------------------------------------------
@@ -137,10 +148,11 @@ end
 local send_message
 
 -- yield the headers all at once, it's faster
-local function send_headers(headers)
+local function send_headers(tosend)
+    local canonic = headers.canonic
     local h = "\r\n"
-    for i,v in base.pairs(headers) do
-        h = i .. ': ' .. v .. "\r\n" .. h
+    for f,v in base.pairs(tosend) do
+        h = (canonic[f] or f) .. ': ' .. v .. "\r\n" .. h
     end
     coroutine.yield(h)
 end
@@ -149,7 +161,7 @@ end
 local function send_multipart(mesgt)
     -- make sure we have our boundary and send headers
     local bd = newboundary()
-    local headers = mesgt.headers or {}
+    local headers = lower_headers(mesgt.headers or {})
     headers['content-type'] = headers['content-type'] or 'multipart/mixed'
     headers['content-type'] = headers['content-type'] ..
         '; boundary="' ..  bd .. '"'
@@ -176,7 +188,7 @@ end
 -- yield message body from a source
 local function send_source(mesgt)
     -- make sure we have a content-type
-    local headers = mesgt.headers or {}
+    local headers = lower_headers(mesgt.headers or {})
     headers['content-type'] = headers['content-type'] or
         'text/plain; charset="iso-8859-1"'
     send_headers(headers)
@@ -192,7 +204,7 @@ end
 -- yield message body from a string
 local function send_string(mesgt)
     -- make sure we have a content-type
-    local headers = mesgt.headers or {}
+    local headers = lower_headers(mesgt.headers or {})
     headers['content-type'] = headers['content-type'] or
         'text/plain; charset="iso-8859-1"'
     send_headers(headers)
@@ -209,20 +221,17 @@ end
 
 -- set defaul headers
 local function adjust_headers(mesgt)
-    local lower = {}
-    for i,v in base.pairs(mesgt.headers or lower) do
-        lower[string.lower(i)] = v
-    end
+    local lower = lower_headers(mesgt.headers)
     lower["date"] = lower["date"] or
-        os.date("!%a, %d %b %Y %H:%M:%S ") .. (mesgt.zone or ZONE)
+        os.date("!%a, %d %b %Y %H:%M:%S ") .. (mesgt.zone or _M.ZONE)
     lower["x-mailer"] = lower["x-mailer"] or socket._VERSION
     -- this can't be overriden
     lower["mime-version"] = "1.0"
-    mesgt.headers = lower
+    return lower
 end
 
-function message(mesgt)
-    adjust_headers(mesgt)
+function _M.message(mesgt)
+    mesgt.headers = adjust_headers(mesgt)
     -- create and return message source
     local co = coroutine.create(function() send_message(mesgt) end)
     return function()
@@ -235,11 +244,13 @@ end
 ---------------------------------------------------------------------------
 -- High level SMTP API
 -----------------------------------------------------------------------------
-send = socket.protect(function(mailt)
-    local s = open(mailt.server, mailt.port, mailt.create)
+_M.send = socket.protect(function(mailt)
+    local s = _M.open(mailt.server, mailt.port, mailt.create)
     local ext = s:greet(mailt.domain)
     s:auth(mailt.user, mailt.password, ext)
     s:send(mailt)
     s:quit()
     return s:close()
 end)
+
+return _M

lualibs/socket/tp.lua

@@ -2,7 +2,6 @@
 -- Unified SMTP/FTP subsystem
 -- LuaSocket toolkit.
 -- Author: Diego Nehab
--- RCS ID: $Id: tp.lua 1418 2006-04-25 09:38:15Z 3rdparty $
 -----------------------------------------------------------------------------
 
 -----------------------------------------------------------------------------
@@ -12,12 +11,14 @@ local base = _G
 local string = require("string")
 local socket = require("socket")
 local ltn12 = require("ltn12")
-module("socket.tp")
+
+socket.tp = {}
+local _M = socket.tp
 
 -----------------------------------------------------------------------------
 -- Program constants
 -----------------------------------------------------------------------------
-TIMEOUT = 60
+_M.TIMEOUT = 60
 
 -----------------------------------------------------------------------------
 -- Implementation
@@ -64,6 +65,7 @@ function metat.__index:check(ok)
 end
 
 function metat.__index:command(cmd, arg)
+    cmd = string.upper(cmd)
     if arg then
         return self.c:send(cmd .. " " .. arg.. "\r\n")
     else
@@ -105,14 +107,14 @@ end
 -- closes the underlying c
 function metat.__index:close()
     self.c:close()
-	return 1
+    return 1
 end
 
 -- connect with server and return c object
-function connect(host, port, create, timeout)
-    local c, e = (create or socket.tcp())
+function _M.connect(host, port, timeout, create)
+    local c, e = (create or socket.tcp)()
     if not c then return nil, e end
-    c:settimeout(timeout or TIMEOUT)
+    c:settimeout(timeout or _M.TIMEOUT)
     local r, e = c:connect(host, port)
     if not r then
         c:close()
@@ -121,3 +123,4 @@ function connect(host, port, create, timeout)
     return base.setmetatable({c = c}, metat)
 end
 
+return _M

lualibs/socket/url.lua

@@ -2,7 +2,6 @@
 -- URI parsing, composition and relative URL resolution
 -- LuaSocket toolkit.
 -- Author: Diego Nehab
--- RCS ID: $Id: url.lua 1418 2006-04-25 09:38:15Z 3rdparty $
 -----------------------------------------------------------------------------
 
 -----------------------------------------------------------------------------
@@ -11,12 +10,15 @@
 local string = require("string")
 local base = _G
 local table = require("table")
-module("socket.url")
+local socket = require("socket")
+
+socket.url = {}
+local _M = socket.url
 
 -----------------------------------------------------------------------------
 -- Module version
 -----------------------------------------------------------------------------
-_VERSION = "URL 1.0"
+_M._VERSION = "URL 1.0.3"
 
 -----------------------------------------------------------------------------
 -- Encodes a string into its escaped hexadecimal representation
@@ -25,10 +27,10 @@ _VERSION = "URL 1.0"
 -- Returns
 --   escaped representation of string binary
 -----------------------------------------------------------------------------
-function escape(s)
-    return string.gsub(s, "([^A-Za-z0-9_])", function(c)
+function _M.escape(s)
+    return (string.gsub(s, "([^A-Za-z0-9_])", function(c)
         return string.format("%%%02x", string.byte(c))
-    end)
+    end))
 end
 
 -----------------------------------------------------------------------------
@@ -40,25 +42,25 @@ end
 --   escaped representation of string binary
 -----------------------------------------------------------------------------
 local function make_set(t)
-	local s = {}
-	for i,v in base.ipairs(t) do
-		s[t[i]] = 1
-	end
-	return s
+    local s = {}
+    for i,v in base.ipairs(t) do
+        s[t[i]] = 1
+    end
+    return s
 end
 
 -- these are allowed withing a path segment, along with alphanum
 -- other characters must be escaped
 local segment_set = make_set {
     "-", "_", ".", "!", "~", "*", "'", "(",
-	")", ":", "@", "&", "=", "+", "$", ",",
+    ")", ":", "@", "&", "=", "+", "$", ",",
 }
 
 local function protect_segment(s)
-	return string.gsub(s, "([^A-Za-z0-9_])", function (c)
-		if segment_set[c] then return c
-		else return string.format("%%%02x", string.byte(c)) end
-	end)
+    return string.gsub(s, "([^A-Za-z0-9_])", function (c)
+        if segment_set[c] then return c
+        else return string.format("%%%02x", string.byte(c)) end
+    end)
 end
 
 -----------------------------------------------------------------------------
@@ -68,10 +70,10 @@ end
 -- Returns
 --   escaped representation of string binary
 -----------------------------------------------------------------------------
-function unescape(s)
-    return string.gsub(s, "%%(%x%x)", function(hex)
+function _M.unescape(s)
+    return (string.gsub(s, "%%(%x%x)", function(hex)
         return string.char(base.tonumber(hex, 16))
-    end)
+    end))
 end
 
 -----------------------------------------------------------------------------
@@ -121,7 +123,7 @@ end
 -- Obs:
 --   the leading '/' in {/<path>} is considered part of <path>
 -----------------------------------------------------------------------------
-function parse(url, default)
+function _M.parse(url, default)
     -- initialize default parameters
     local parsed = {}
     for i,v in base.pairs(default or parsed) do parsed[i] = v end
@@ -142,7 +144,7 @@ function parse(url, default)
         parsed.authority = n
         return ""
     end)
-    -- get query stringing
+    -- get query string
     url = string.gsub(url, "%?(.*)", function(q)
         parsed.query = q
         return ""
@@ -158,9 +160,12 @@ function parse(url, default)
     if not authority then return parsed end
     authority = string.gsub(authority,"^([^@]*)@",
         function(u) parsed.userinfo = u; return "" end)
-    authority = string.gsub(authority, ":([^:]*)$",
+    authority = string.gsub(authority, ":([^:%]]*)$",
         function(p) parsed.port = p; return "" end)
-    if authority ~= "" then parsed.host = authority end
+    if authority ~= "" then 
+        -- IPv6?
+        parsed.host = string.match(authority, "^%[(.+)%]$") or authority 
+    end
     local userinfo = parsed.userinfo
     if not userinfo then return parsed end
     userinfo = string.gsub(userinfo, ":([^:]*)$",
@@ -177,24 +182,27 @@ end
 -- Returns
 --   a stringing with the corresponding URL
 -----------------------------------------------------------------------------
-function build(parsed)
-    local ppath = parse_path(parsed.path or "")
-    local url = build_path(ppath)
+function _M.build(parsed)
+    local ppath = _M.parse_path(parsed.path or "")
+    local url = _M.build_path(ppath)
     if parsed.params then url = url .. ";" .. parsed.params end
     if parsed.query then url = url .. "?" .. parsed.query end
-	local authority = parsed.authority
-	if parsed.host then
-		authority = parsed.host
-		if parsed.port then authority = authority .. ":" .. parsed.port end
-		local userinfo = parsed.userinfo
-		if parsed.user then
-			userinfo = parsed.user
-			if parsed.password then
-				userinfo = userinfo .. ":" .. parsed.password
-			end
-		end
-		if userinfo then authority = userinfo .. "@" .. authority end
-	end
+    local authority = parsed.authority
+    if parsed.host then
+        authority = parsed.host
+        if string.find(authority, ":") then -- IPv6?
+            authority = "[" .. authority .. "]"
+        end
+        if parsed.port then authority = authority .. ":" .. parsed.port end
+        local userinfo = parsed.userinfo
+        if parsed.user then
+            userinfo = parsed.user
+            if parsed.password then
+                userinfo = userinfo .. ":" .. parsed.password
+            end
+        end
+        if userinfo then authority = userinfo .. "@" .. authority end
+    end
     if authority then url = "//" .. authority .. url end
     if parsed.scheme then url = parsed.scheme .. ":" .. url end
     if parsed.fragment then url = url .. "#" .. parsed.fragment end
@@ -210,14 +218,14 @@ end
 -- Returns
 --   corresponding absolute url
 -----------------------------------------------------------------------------
-function absolute(base_url, relative_url)
+function _M.absolute(base_url, relative_url)
     if base.type(base_url) == "table" then
         base_parsed = base_url
-        base_url = build(base_parsed)
+        base_url = _M.build(base_parsed)
     else
-        base_parsed = parse(base_url)
+        base_parsed = _M.parse(base_url)
     end
-    local relative_parsed = parse(relative_url)
+    local relative_parsed = _M.parse(relative_url)
     if not base_parsed then return relative_url
     elseif not relative_parsed then return base_url
     elseif relative_parsed.scheme then return relative_url
@@ -238,7 +246,7 @@ function absolute(base_url, relative_url)
                     relative_parsed.path)
             end
         end
-        return build(relative_parsed)
+        return _M.build(relative_parsed)
     end
 end
 
@@ -249,17 +257,17 @@ end
 -- Returns
 --   segment: a table with one entry per segment
 -----------------------------------------------------------------------------
-function parse_path(path)
-	local parsed = {}
-	path = path or ""
-	--path = string.gsub(path, "%s", "")
-	string.gsub(path, "([^/]+)", function (s) table.insert(parsed, s) end)
-	for i = 1, table.getn(parsed) do
-		parsed[i] = unescape(parsed[i])
-	end
-	if string.sub(path, 1, 1) == "/" then parsed.is_absolute = 1 end
-	if string.sub(path, -1, -1) == "/" then parsed.is_directory = 1 end
-	return parsed
+function _M.parse_path(path)
+    local parsed = {}
+    path = path or ""
+    --path = string.gsub(path, "%s", "")
+    string.gsub(path, "([^/]+)", function (s) table.insert(parsed, s) end)
+    for i = 1, #parsed do
+        parsed[i] = _M.unescape(parsed[i])
+    end
+    if string.sub(path, 1, 1) == "/" then parsed.is_absolute = 1 end
+    if string.sub(path, -1, -1) == "/" then parsed.is_directory = 1 end
+    return parsed
 end
 
 -----------------------------------------------------------------------------
@@ -270,28 +278,30 @@ end
 -- Returns
 --   path: corresponding path stringing
 -----------------------------------------------------------------------------
-function build_path(parsed, unsafe)
-	local path = ""
-	local n = table.getn(parsed)
-	if unsafe then
-		for i = 1, n-1 do
-			path = path .. parsed[i]
-			path = path .. "/"
-		end
-		if n > 0 then
-			path = path .. parsed[n]
-			if parsed.is_directory then path = path .. "/" end
-		end
-	else
-		for i = 1, n-1 do
-			path = path .. protect_segment(parsed[i])
-			path = path .. "/"
-		end
-		if n > 0 then
-			path = path .. protect_segment(parsed[n])
-			if parsed.is_directory then path = path .. "/" end
-		end
-	end
-	if parsed.is_absolute then path = "/" .. path end
-	return path
+function _M.build_path(parsed, unsafe)
+    local path = ""
+    local n = #parsed
+    if unsafe then
+        for i = 1, n-1 do
+            path = path .. parsed[i]
+            path = path .. "/"
+        end
+        if n > 0 then
+            path = path .. parsed[n]
+            if parsed.is_directory then path = path .. "/" end
+        end
+    else
+        for i = 1, n-1 do
+            path = path .. protect_segment(parsed[i])
+            path = path .. "/"
+        end
+        if n > 0 then
+            path = path .. protect_segment(parsed[n])
+            if parsed.is_directory then path = path .. "/" end
+        end
+    end
+    if parsed.is_absolute then path = "/" .. path end
+    return path
 end
+
+return _M

packages/sample.lua

@@ -27,9 +27,10 @@ local events = {
   onEditorKeyDown =    function(self, editor, event) end, -- return false
   onEditorCharAdded =  function(self, editor, event) end, -- return false
   onMenuEditor =       function(self, menu, editor, event) end,
-  onMenuEditorTab =    function(self, menu, notebook, event) end,
+  onMenuEditorTab =    function(self, menu, notebook, event, index) end,
   onMenuFiletree =     function(self, menu, tree, event) end,
-  onProjectLoad =      function(self, project) end,
+  onProjectPreLoad =   function(self, project) end, -- before project is changed
+  onProjectLoad =      function(self, project) end, -- after project is changed
   onProjectClose =     function(self, project) end,
   onInterpreterLoad =  function(self, interpreter) end,
   onInterpreterClose = function(self, interpreter) end,
@@ -65,8 +66,7 @@ local events = {
     DisplayOutputLn(self:GetFileName(), "onMenuEditor")
   end
 
-  P.onMenuEditorTab = function(self, menu, notebook, event)
-    local index = event:GetSelection()
+  P.onMenuEditorTab = function(self, menu, notebook, event, index)
     menu:Append(id, ">> Sample item; tab "..index)
     menu:Enable(id, true)
 

spec/cg.lua

@@ -19,27 +19,19 @@ return {
   end,
 
   lexerstyleconvert = {
-    text = {wxstc.wxSTC_C_IDENTIFIER,
-      wxstc.wxSTC_C_VERBATIM,
-      wxstc.wxSTC_C_REGEX,
-      wxstc.wxSTC_C_REGEX,
-      wxstc.wxSTC_C_GLOBALCLASS,},
+    text = {wxstc.wxSTC_C_IDENTIFIER,},
 
     lexerdef = {wxstc.wxSTC_C_DEFAULT,},
     comment = {wxstc.wxSTC_C_COMMENT,
       wxstc.wxSTC_C_COMMENTLINE,
-      wxstc.wxSTC_C_COMMENTDOC,
-      wxstc.wxSTC_C_COMMENTLINEDOC,
-      wxstc.wxSTC_C_COMMENTDOCKEYWORD,
-      wxstc.wxSTC_C_COMMENTDOCKEYWORDERROR,},
+      wxstc.wxSTC_C_COMMENTDOC,},
     stringtxt = {wxstc.wxSTC_C_STRING,
       wxstc.wxSTC_C_CHARACTER,
-      wxstc.wxSTC_C_UUID,},
+      wxstc.wxSTC_C_VERBATIM,},
     stringeol = {wxstc.wxSTC_C_STRINGEOL,},
     preprocessor= {wxstc.wxSTC_C_PREPROCESSOR,},
     operator = {wxstc.wxSTC_C_OPERATOR,},
-    number = {wxstc.wxSTC_C_NUMBER,
-      wxstc.wxSTC_C_WORD},
+    number = {wxstc.wxSTC_C_NUMBER,},
 
     keywords0 = {wxstc.wxSTC_C_WORD,},
     keywords1 = {wxstc.wxSTC_C_WORD2,},

spec/glsl.lua

@@ -83,6 +83,7 @@ return {
     gl_NumWorkGroups gl_WorkGroupSize gl_WorkGroupID gl_LocalInvocationID gl_GlobalInvocationID gl_LocalInvocationIndex
     local_size_x local_size_y local_size_z
     gl_BaseVertexARB gl_BaseInstanceARB gl_DrawIDARB
+    bindless_sampler bound_sampler bindless_image bound_image
 
     coherent volatile restrict readonly writeonly
     image1D image2D image3D image2DRect imageCube imageBuffer image1DArray image2DArray imageCubeArray image2DMS image2DMSArray