Extending Lua scripting with your own code

Posted by Nick Gammon on Wed 24 Nov 2004 07:28 PM — 40 posts, 167,646 views.

Australia Forum Administrator #0
Lua has a provision for extending it by writing your own DLLs, thus making anything possible (eg. opening Windows dialogs etc.).

A brief example follows. The first thing to note is that every function called from Lua has an identical calling pattern:


static int function_name (lua_State *L)
  {
  /* do something here */  
  return 0;   /* number of results returned */
  } /* end of function_name */


You use this "lua state" called L (although you can call it anything of course) to access all Lua data and functions, and to return results.

As an example I will write a small function that calculates miles to kilometres, although in practice you would simply do that in Lua.


static int miles_to_km (lua_State *L)
  {
  double miles = luaL_checknumber (L, 1);
  double km = miles * 1.609;
  lua_pushnumber (L, km);
  return 1;   /* one result */
  } /* end of miles_to_km */


The first thing the above does is extract the first argument passed to the function into "miles". If I passed a second argument it would be accessed with (L, 2) instead of (L, 1), and so on.

Then I do some calculations on it.

Finally the result is "pushed" onto the Lua stack (lua_pushnumber). Then the function returns the number 1, to tell Lua that one result is to be returned to the caller.

To make this into a DLL we need a little bit of infrastructure around it. From Lua, when we load a DLL we get to call a single entry point, so if we want to expose more than one function, we need to make a "library" of them. The code below has two functions, the miles-to-kilometre one, and another that calculates the circumference and area of a circle. This second one illustrates returning multiple results:


static int circle_calcs (lua_State *L)
  {
  double radius = luaL_checknumber (L, 1);
  double circumference = radius * 2 * PI;
  double area = PI * radius * radius;
  lua_pushnumber (L, circumference);
  lua_pushnumber (L, area);
  return 2;   /* two results */
  } /* end of miles_to_km */


Next thing we need to do is "register" these functions with Lua, so it knows to add them to its address space. This is done with a small table, that maps function names to their addresses, and then a library call:


static const luaL_reg testlib[] = 
{
  {"miles_to_km", miles_to_km},
  {"circle_calcs", circle_calcs},
  {NULL, NULL}
};


/*
** Open test library
*/
LUALIB_API int luaopen_test (lua_State *L)
 {
  luaL_openlib(L, "test", testlib, 0);
  return 1;
 }


The function luaopen_test will be our entry point for the DLL. When called it adds miles_to_km and circle_calcs to the "test" library. After that you will be able to do:


k = test.miles_to_km (42)
c, a = test.circle_calcs (16)


We will wrap it all up with a couple of include files, and a definition for LUA_API, which exports the function when the DLL is built:

test.c



#ifdef _WIN32
#define LUA_API __declspec(dllexport)
#endif

#pragma comment( lib, "lua.lib" )
#pragma comment( lib, "lualib.lib" )

#include "lua.h"

#include "lauxlib.h"
#include "lualib.h"


#define PI (3.14159265358979323846)

static int miles_to_km (lua_State *L)
  {
  double miles = luaL_checknumber (L, 1);
  double km = miles * 1.609;
  lua_pushnumber (L, km);
  return 1;   /* one result */
  } /* end of miles_to_km */

static int circle_calcs (lua_State *L)
  {
  double radius = luaL_checknumber (L, 1);
  double circumference = radius * 2 * PI;
  double area = PI * radius * radius;
  lua_pushnumber (L, circumference);
  lua_pushnumber (L, area);
  return 2;   /* one result */
  } /* end of miles_to_km */

static const luaL_reg testlib[] = 
{
  {"miles_to_km", miles_to_km},
  {"circle_calcs", circle_calcs},
  {NULL, NULL}
};


/*
** Open test library
*/
LUALIB_API int luaopen_test (lua_State *L)
 {
  luaL_openlib(L, "test", testlib, 0);
  return 1;
 }



The pragma instructions above tell the compiler to link against the lua.lib and lualib.lib files, which expose the entry points in the lua DLLs.

Compile that under Visual C++ 6, and we get a test.dll file output (1.86 Kb, pretty small, huh?).

Copy that dll to the same directory as where MUSHclient is (or our Lua executable is) and we are ready to test.

Start up Lua, or use Lua scripting in MUSHclient.

Next, we load the DLL:


f, e1, e2 = loadlib ("test.dll", "luaopen_test")


This loads our nominated DLL, and tells it to return the exposed entry point luaopen_test as a function.

We can then test how that went:


print (f, e1, e2)  --> function: 0062FB50 nil nil


With a bit of luck f will be a function, and the other two arguments (error messages) will be nil. That means it worked.

However if you see this:


nil The specified module could not be found.
 open


Then that means the function f was not loaded (is nil) and an error message, and error reason (open). This might be because you didn't put the DLL where it could be found (or the lua.dll or lualib.dll were not found either).

Assuming it loaded OK, then you call the function to actually register the routines in the library:


f ()


After you have done that, you can now use the "test" library.


print (test.miles_to_km (40))  --> 64.36
print (test.circle_calcs (15)) --> 94.247779607694 706.8583470577


Note in the second case how we see printed both arguments that were returned. If we want to store them, we could do this:


c, a = test.circle_calcs (15)






[EDIT]

A better way of loading the library is to use "assert" - this checks for a non-nil result. If nil it raises an error giving the error message, if not nil it returns the result. So, the whole load process can be done on one line.

Rather than:


f, e1, e2 = loadlib ("test.dll", "luaopen_test")
if not f then
  error (e1)
end -- if
f ()


Do this:


assert (loadlib ("test.dll", "luaopen_test")) ()


The assert assures us we have a function, and the final set of brackets runs the function, thus installing the library.
Amended on Sun 14 May 2006 05:47 AM by Nick Gammon
Australia Forum Administrator #1
Compiling under Cygwin

I compiled exactly the same file above using Cygwin, with this command line:


gcc -shared -o test.dll test.c -L/usr/local/lib/ -llua -llualib


The only real trick was to specify the location of the files liblua.a liblualib.a which are the Cygwin versions of the libraries to link against. To get these files in the first place you need to download and install the Lua source under Cygwin.




[EDIT]

Actually, a better way is this:


gcc  -mno-cygwin -shared -o test.dll test.c lualib.lib lua.lib


This does not require you to compile Lua, because it uses the libraries generated by Visual C++ as part of the DLL compilation. These libraries are in the standalone download from this site (the files you need are lualib.lib and lua.lib).

This method produces a DLL that is 12.9 Kb, compared to 104 Kb using the first method. I think the main reason is the inclusion of the Cygwin libraries, which are not needed in this case.
Amended on Wed 24 Nov 2004 09:18 PM by Nick Gammon
Greece #2
Hmm... Could the DLL call a Lua function that in turn would do something in MC (effectively a callback)?
Australia Forum Administrator #3
Now for another example, I'll call a Windows API - MessageBox, to display a GUI box on the screen.



#include <windows.h>

#define LUA_API __declspec(dllexport)

#pragma comment( lib, "lua.lib" )
#pragma comment( lib, "lualib.lib" )

#include "lua.h"

#include "lauxlib.h"
#include "lualib.h"


static int messagebox (lua_State *L)
  {
  const char * m = luaL_checkstring (L, 1);

  MessageBox (0, m, "Message", MB_OK);
  return 0;   /* no results */
  } /* end of messagebox */


static const luaL_reg msglib[] = 
{
  {"messagebox", messagebox},
  {NULL, NULL}
};


/*
** Open msg library
*/
LUALIB_API int luaopen_msglib  (lua_State *L)
 {
  luaL_openlib(L, "msg", msglib, 0);
  return 1;
 }



Compile and link this the same way.

Now to test it. Load and install the library:


loadlib ("test.dll", "luaopen_msglib") ()


This loads the library and calls the function in a single line (that's those final parentheses).

Now display a message box:


msg.messagebox "hi there"


A little box pops up with "hi there" in it.
Australia Forum Administrator #4
Quote:

Could the DLL call a Lua function that in turn would do something in MC (effectively a callback)?


Yes it can. This will illustrate it. I have taken my "messagebox" code and added another function "callback".

This will take two arguments:

  • A string to be displayed
  • A count in the range 1 to 10


It will display the string "n" times, using ColourNote - a MUSHclient function. This is the callback you are talking about.



#include <windows.h>

#define LUA_API __declspec(dllexport)

#pragma comment( lib, "lua.lib" )
#pragma comment( lib, "lualib.lib" )

#include "lua.h"

#include "lauxlib.h"
#include "lualib.h"


static int messagebox (lua_State *L)
  {
  const char * m = luaL_checkstring (L, 1);

  MessageBox (0, m, "Message", MB_OK);
  return 0;   /* no results */
  } /* end of messagebox */

static int callback_test (lua_State *L)
  {
  const char * m = luaL_checkstring (L, 1);
  int n = (int) luaL_checknumber (L, 2);
  int i;

  if (n < 1 || n > 10)
    luaL_error (L, "count out of range");

    for (i = 0; i < n; i++)
    {
    lua_getglobal (L, "world");   /* world functions */
    lua_pushliteral (L, "ColourNote"); /* ColourNote function */
    lua_gettable (L, -2);  /* get function */
    lua_pushliteral (L, "white");  /* foreground colour */
    lua_pushliteral (L, "blue");   /* background colour */
    lua_pushstring  (L, m);  /* message */
    lua_call (L, 3, 0);  /* call with 3 args and no result */
    }


  return 0;   /* no results */
  } /* end of callback_test */


static const luaL_reg msglib[] = 
{
  {"callback", callback_test},
  {"messagebox", messagebox},
  {NULL, NULL}
};


/*
** Open msg library
*/
LUALIB_API int luaopen_msglib  (lua_State *L)
 {
  luaL_openlib(L, "msg", msglib, 0);
  return 1;
 }



The important part is the part in bold. It pushes the name of the wanted function ("ColourNote") and asks for it from the global address space. Then it pushes the three arguments for ColourNote and does a lua_call. This does the callback.

You would test the above like this:


  loadlib ("test.dll", "luaopen_msglib") ()
  msg.callback ("test", 3)

Amended on Tue 13 Dec 2005 10:15 PM by Nick Gammon
Australia Forum Administrator #5
Of course, you are not limited to simply calling MUSHclient internal functions. You can call *any* function that is in the same script space, including Lua ones, or any user-written functions that you have already added into the script.
Australia Forum Administrator #6
A couple of caveats to the above process. I found that under Cygwin (or indeed standalone) that if the DLLs needed by Lua were not found, you get the error message "The specified module could not be found." when trying to load your DLL.

This is a bit confusing, because you think it can't find "test.dll" (or whatever you have called it).

One way of working this out is to right-click on the DLL and selecting "View Dependencies". This will show which other DLLs are required.

The next thing I wanted to do was make a shared library under Linux, to show how you could also write libraries if you are using Linux instead of Windows.

First, I had to change the first line in the library from:


#define LUA_API __declspec(dllexport)


to:


#ifdef _WIN32
#define LUA_API __declspec(dllexport)
#endif


Otherwise, you get an error message about "__declspec".

(I have done this now in the original post, to save confusion.)

Next, you can compile like this:


gcc -shared -o test.so test.c


This gives a file test.so (shared object). To load and test this library I used this line (after typing "lua" to invoke the stand-alone version):


assert (loadlib ("./test.so", "luaopen_test")) ()
print (test.miles_to_km (40))  --> 64.36
print (test.circle_calcs (15)) --> 94.247779607694 706.8583470577


In this case I used "./test.so" as the path to the library, as I suspect that the current directory was not in the library search path.
Amended on Tue 13 Dec 2005 10:17 PM by Nick Gammon
Canada #7
Your post helped me alot, but I have a problem. I'm writing a small wrapper around jfwapi.dll (a dll that is used by jaws (screen reader) to speak text). It works fine inside the lua shell. When I load it into MC, it can't find the dll. It finds the jfw.dll (library), but it can't find jfwapi.dll, which I put inside the mushclient directory. What should I do to tell the library where to find its dll file? trying to statically link the dll in didn't work. I think it has something to do with the current working directory, but if loadlib can find the file, why can't windows find jfwapi.dll in the same place?
Australia Forum Administrator #8
You aren't simply getting a problem with loadlib are you? That is disabled by default in the MUSHclient sandbox.

Assuming that is not the problem, I'm not sure the exact way that Windows searches for DLLs. I think it may be specified in the PATH environment variable. To see its value try this:


print (os.getenv ("PATH"))


Amongst other things I get this:


C:\WINDOWS\system32;C:\WINDOWS


Thus, you could try putting the jaws DLL in one of those locations, such as C:\WINDOWS\system32.
USA #9
Interesting..

This brings up something else I have been reading up on recently. You know that whole event issue. Seems MFC, ATL and script systems, as well as languages like VB, all "hide" the mechanisms actually in use. What is interesting is that "assert" is providing this:

a = createobject("test.dll") <- Loadlib
b = a.QueryInterface(IID_luaopen_test, myptr)

If b = OK, then myptr contains the function pointer for that call.

Now, there are also functions in ActiveX components, missing from standard dlls, which allow you to enumerate its other interfaces.

What the problem with events seems to be is that MFC, ATL, VB, scripts, etc. all "hide" this though stuff like "assert", but they also assume that you won't be creating objects late, then wanted to have them tell you something. These commands are not just buried, they are missing/ignored completely:

GetConnectionInterface - Gets the ID of a specific connection point.
GetConnectionPointContainer - Returns a pointer to the nest of connection points for a known connection, so you can then get the object it belongs to.
Advise - This is used to tie a function in "your" program/script to the outgoing events.
Unadvise - This disconnects that event.
EnumConnections - This lists the events available in the control.

These are not part of QueryInterface, which is what is needed to do "most" of the stuff you do with objects, which is apparently why they can get by with preventing you from accessing these. And this is the "raw" OLE, that "must" exist in all ActiveX components that support events, so unlike the ATL mess I found before, these "can't" change, without breaking ActiveX itself. The only thing I am not 100% sure of is what happens if you fail to call Unadvise before an object is killed or something like a script reload happens (which can end up leaving an active object in memory). I think useing the scripts own "createobject" function might be a bit more dangerous than we realize. Mushclient is unaware of these things and I am not sure what might happen if a loose object tries to call a function, for which the pointer is no longer valid, when it fires its events. Unless the engine itself automatically kills such objects in a reload... But that might not be a major problem. Since Mushclient would be providing the Advise/Unadvise function, a reload could trigger Mushclient to Unadvise any events for that script/plugin, before reloading the script.

Anyway, having read into a lot of what makes the whole mess work, I found it interesting how this thread tied into the whole setup of how just general DLLs work, never mind ActiveX. Libraries that are not statically linked "must" contain the same basic intefaces to work, and obviously something in the "loadlib" or "assert" process is using QueryInterface.
USA #10
Dynamic linking is a very interesting topic and an extremely powerful technique. QueryInterface etc. is for COM stuff; the real dynamic linking is done by the OS. Basically, the host application (the one asking for the DLL) expects to find certain symbols in the DLL, and the DLL can also expect certain symbols in the host. If any of these symbols are not found, the linking fails.
Amended on Wed 05 Jul 2006 07:05 PM by David Haley
Canada #11
Basically, I wanted to make the speech interface transparent. put a plugin in the plugin folder, 2 dll files in the mushclient main directory, and that's it. Seems I have to either 1) remove the start in field from the shortcut (which might work, haven't tested - ony until Mushclient changes with cwd, of course), or get people to add the mushclient directory to the path environment variable. Is there any reason for the working directory to change frequently?
USA #12
Not sure why the working directory would change, unless Nick does it somewhere in the code.

If all you want to do is guarantee access to the DLLs, you could just stick them into the Windows System32 directory. It's not the cleanest of solutions (because if you remove MUSHclient, the DLLs "litter" the directory) but it would work fairly well.

Lua 5.1 lets you specify where to look for C libraries before including them, which seems like it would help solve your problem. But MUSHclient uses the 5.0 release, if I'm not mistaken. If the compat-5.1 library was used, you could do something similar and the DLLs could be wherever you wanted. (But, you'd have to see if you could convince MUSHclient to load it up.)
Australia Forum Administrator #13
Quote:

Basically, I wanted to make the speech interface transparent. put a plugin in the plugin folder, 2 dll files in the mushclient main directory, and that's it.


And, get them to change the sandbox which by default doesn't allow loadlib. Having done that you could perhaps also add in support for doing a "cd" to make sure the directory is where you think. I'm not sure why you are having these problems, MUSHclient successfully finds the lua DLL and the spellcheck DLL for most people. Sounds fishy. You know that if the jfwapi.dll has a missing dependency then it will not "find" it? So maybe there is yet another DLL missing.
Australia Forum Administrator #14
Quote:

What is interesting is that "assert" is providing this:

a = createobject("test.dll") <- Loadlib
b = a.QueryInterface(IID_luaopen_test, myptr)

...

... MFC, ATL, VB, scripts, etc. all "hide" this though stuff like "assert" ...



This is not all strictly correct, sorry Shadowfyr. For one thing, "assert" is not providing anything, it simply checks for an error return.

eg.


assert (2 + 2 == 4) --> no error
assert (2 + 2 == 5) --> assertion failed!


I am assuming you meant "loadlib is providing this ...".

The next part is you are confusing the COM (Component Object Model) interface with a simple DLL (Dynamic Link Library).

A DLL does not necessarily use COM, and the ones on this post don't.

The example DLL that starts this thread consists of a number of functions, one of which (luaopen_test) is "exported" (made publicly available) by the use of the keyword __declspec(dllexport).

When you load the library like this:


f = loadlib ("test.dll", "luaopen_test")


... 2 conditions need to be met. First is that test.dll is found in the search path (and is a DLL of course) and the second is that the entry point named "luaopen_test" is found.

In this case if f is non-nil then we have been given a function pointer, and we can now call it:



f ()


This executes the function body which in this case does this:


luaL_openlib(L, "test", testlib, 0);


What this does is add the items in the testlib table to a lua table called "test" in the Lua address space.




COM does similar things but in their own way. Like the Lua example, a DLL using COM would have an internal table of functions, and this is what gets used when you do QueryInterface - it finds if a function exists. However this is by function number (eg. function 1, function 2, etc.).

You also have to use IDispatch.GetIDsOfNames to map a name to an ID. I am not a great expert on COM, but I think you need to use the generic interface (IUnknown) to find IDispatch, and then (if found) use IDispatch interface GetIDsOfNames to eventually map a name to an ID.

Finally you can retrieve that interface. A lot more complicated than the Lua way (or using a straight DLL entry point).

Quote:

These are not part of QueryInterface, which is what is needed to do "most" of the stuff you do with objects, which is apparently why they can get by with preventing you from accessing these


I'm not totally sure what you mean by this, but in my example the "raw" functions (like miles_to_km) are not directly exposed (the 'static' keyword makes sure of that), so you cannot directly access miles_to_km from the DLL.

Thus, this would fail:


f = loadlib ("test.dll", "miles_to_km")


This is by design, as in Lua you don't want the pure function, what you really want is the function to be added to the Lua address space, which is what the one exposed function (luaopen_test) does for you.

In a similar way, QueryInterface in COM gives you back the interface, thus it is QueryInterface that needs to be exposed in the DLL, not the raw function.
USA #15
Hmm. Yeah. After some further consideration, you're right. And dlls need to use some "similar" standard method, otherwise you couldn't link them using something like Lua. Without a static or some dynamic way to gain it at runtime, there is no way to determine the entry points at all. Some method is needed to tell the instancing program where the entry points are, even if its not the identical one of QueryInterface. IDispatch is, so far, not even mentioned. This tends to make me suspect that it is an MFC function, not a direct connection to QueryInterface. In essense, when you use the IDispatch methods you are letting MFC handle the dirty work of finding and returning the entry points. This is fine, except MFC doesn't provide a parallel method to the connection point system, nor does it allow "direct" means of handling some, so called, design type properties, which may in some case be useful to have a way to adjust.

Anyway. My main point is that after several false starts, I think I now have a fairly clear idea what the missing bits are that are needed to handle events. Its all in the OLE system. The problem is, you have to use raw C++ to code it, well, besides the OLE parts, because MFC, ATL and nearly every "high level language", buries the functionality behind its libraries, where you can't get to it and don't even know that it exists. Heck, I don't think you can even enumerate properties or interfaces in MFC, without resorting to purer COM functions.

Yeah, I kind of confused things a bit. I took the opertunity to cross over to another subject that I had brought up several times before, without "quite" knowing what I was trying to actually do. I went through two books, ATL Internals, trying to find the ATL solution, which I didn't, and then Essential COM, by Don Box, which helped a lot, but tended to go "way" to much into the theory of how it worked, without quite providing practical examples and tended to gloss over the stuff I wanted to find out about. I am now reading Inside Ole, which unfortunately was written by MS. And if you know anything about how MS writes things, you know that practical examples, instead of comvoluted techinical explanations that can sometimes confuse you more than you started, isn't exactly high on there list. Its like reading a history book, with all the dates and general themese intact, but no names of places, people or context to identify whose history it is refering too. lol

Well, OK, not that bad, but without a compiler to fiddle with the code in, its not as useful as one concrete example, or even some, "Here is a simple program that does X and uses everything you will learn in this book. Now lets see how to do it.", type book. Its more like, "Here are fifty 5-20 line examples, program scraps and fragments of things that don't "quite" do anything. Hopefully you can use this to write a coherent application that *actually* does do something." ;) I am not sure which is worse, the boredom or the incoherent way it skips from one subject to the next, without bothering to "quite" connect it all together. lol
USA #16
No, really, the linking is done by the operating system. What MFC and COM and all that provide is a higher level of abstraction, where you only need to know of one entry point, and from there you can explore the other "exposed" interfaces. But the interfaces are exposed on very different levels.

Standard DLLs on Linux/Unix, for instance, (and, again, on Windows) have none of this MFC/COM/ATL/etc. stuff; the programs simply leave symbols unresolved at compiler-link time, and the symbols are resolved by the operating system (well, the C runtime library might be more accurate) when it loads up the DLL for dynamic linking.

For standard DLLs the linking is also "hidden" (normal quotes, not weird emphasis quotes :P). That might be why you aren't finding it... you won't find it anywhere in the code, because it's a service the environment provides for you.
USA #17
Sigh. pp59 Paragraph 3, of Inside OLE: "First of all, no OLE tool I know, even MFC, has direct and explicit support for absolutely every OLE feature that we'll be covering."

Straight from the horses mouth (its an MS published book). Abstraction layers, whether things like MFC, or higher languages, "Do not support", everything that OLE does. I am not finding it, because unless you use OLE directly, it is not available. Hidden in this case means, "You can't use that at all." MFC doesn't expose connection points at all, ATL does, but you have to do some convoluted things to make them work, *neither* support access to design time properties or user_mode, other than to read it, so you can't make a form that lets you drag and drop objects, at least without faking the whole process and adding your own drag handles. And that is a bit like strapping your inflatable boat to a dingy, because you can't figure out how to inflate it, but you still want to use it to row to across the bay.

I know there just provide different levels of abstraction. The whole point of my research is to figure out how to get around the stuff those abstraction layers "don't allow", so some things in Mushclient can be a) easier, b) possible at all in some cases, or c) done without the user doing the same sort of research I have had to, in order to figure it out in the first place.
USA #18
I think you didn't quite see what I meant. I'm talking about something much more fundamental than MFC, OLE, etc. DLLs use technology that really, honestly, is quite independent of OLE. It just so happens that OLE is basically (emphasis on basically) a layer of abstraction on top of DLLs, and COM etc. are a different layer of abstraction.

Would it be possible for you to give a very short summary of what exactly you're trying to do? I get a little confused reading through your long posts because you mix a lot of terminology and it's hard to follow. Maybe if we keep this really straightforward, things will start coming together.

For example you are using 'linking' when speaking of DLLs in a way that is not appropriate. Linking, as done by DLLs, is something very particular that is independent of all these other techniques. When words are used in ways that aren't their conventional meaning, it's hard to follow.
USA #19
Ok, first, lets drop the whole bloody DLL vs ActiveX stuff. That was "never" the intent of what I was saying, beyond mentioning the similarities, which I only recently have come to understand to a reasonable degree. I simply used the similarity to sideline into another issue.

That said, I take it you haven't read "any" of my other posts on the subject I am trying to talk about? Lets try to put this simply then.. Right now there are three ways to create windows, mappers, or any other feature, and have them talk to Mushclient.

1. Link the TLB for Mushclient into them and use CallPlugin.
2. Use UDP.
3. Create a ghost world and use TCP/IP.

The problem with "all" of these solutions is two fold. First, you can't use a control or program you did not **specifically** write for Mushclient, unless its "only" interfaces are one way. I.e. You can, like a database, pass them the "location" you want the response to go in, if they respond, but not let them respond "later", when they are done with some task. Second, they are all basically work arounds, with CallPlugin being the only one that comes even "close" to letting the applications talk directly to your script. This means that existing applications or controls are unusable, if they expect the client that asks to use them to get their response directly, through and event. Only those you can create a sort of Mushclient aware shell to fit in, or you stick on an application of your own (basically the same thing), can talk to Mushclient, indirectly.

This also has another side effect. Anything like custom windows, new notepad windows, etc., all have to be included "in" Mushclient itself, even of some languages, including Lua, make handling those thing from the script trivial, if often requiring, the right addon libraries. Why? Because even "with" those libraries, there is no way to link something as simple as a button click to your script. The script simply cannot ever tell the connection point, "When someone clicks this button, call this script function."

Think about it. That leaves *huge* gap in what you can do. It also means that any potential solutions have to be hard coded into Mushclient, which leaves Nick as the one that has to decide how they should work, not the people that get the idea in the first place, who are not always happy with the solution Nick comes up with.

What I would like to be able to do is create a plugin that can do roughly the following:

win = createobject("Some window object")
win.parent = GetFrame
button1 = createobject("VBButton")
button1.parent = win
button1.lable = "Press me!"
...
Advise ("Button1_Click", "OnBut1_Click")

function OnBut1_Click ()
  note "Someone clicked me!!"
end function

In other words, skip the whole insane mess needed to indirectly make this work. Have the plugin itself define, say, a window that contains icons, to keep track of your spells, let the script respond to clicks on them that are intended to recast the spell, etc. And **not** require that every poor user that wants to do this has to buy a $600 compiler and either dozens of books on how to do anything, or get a college degree, just to write a completely seperate program, which then has to be installed some place, to stick a few pictures in a window. Imho, this is completely rediculous. And it limits even what those of us that can do these things are able to do with the client, because we spend almost as much time trying to figure out how the program should talk to the script as we do designing the program.

A good example of this is the recent mapper. Why? Because I am sure the zMud mapper uses DLL interfaces of ActiveX, which means it "should be" possible to tie it into a Mushclient script, but not if all out-going intefaces are disabled, because the script can't connect them. The solution will invariably be so Mushclient centric that its non usable for anything else, or so generalized, that we have to resort to TCP/IP or UDP to make the connections instead, one of which doesn't always work well, and the other of which requires a imho bizzare hack that, while it works, simply clutters the client with extra worlds that are not invovled in "any" direct game playing.

However, more to the point.. I wouldn't mind seeing, as part of these plugins, something that could read in a "description" of the window you want and another "design" plugin that uses the user_mode switch for its window, so you can *actually* create, place, resize, rename, relabel, etc. the controls you want, save the result to a file, and then import it to your new plugin to provide a pre-designed window of any kind you want. That also requires more direct access to the controls, since all MFC and ATL solutions "assume" that the code that does this is referring to "the window that the code itself belongs too." Or at least, there is no indication how you tell it to change that for any window except the active one, which we absolutely don't have to have happen. Imagine Every window, control and button in Mushclient's main window suddenly becoming movable... while at the same time "all" of the windows controls, buttons, menus, etc. are diabled and stop working. Very bad idea... lol

So, what we need is:

1. Some way to use IUknown::Advise to link a script to the events.
2. Some way to access Ambient_UserMode, but limit it to only script created windows.

Having those, we can create a form designer plugin *and* build damn near anything we want "in" the client, instead of hoping someone with a Bachelors degree, or better, in programming, and the money to buy the tools, decides to bother making some gadget everyone would like to have.

This is in essense what I would like to see be possible. It bugs me that the script languages "can" be extended with readily available libraries for a lot of it, but two relatively minor limitations prevent it from working at all. Or that when Nick does agree to add something, its by hardcoding it into the client, when imho that should be necessary, or for that matter, always desirable.
USA #20
I find it very hard to read your posts like this one and understand what you mean, but it seems that what you really are asking for is to have MUSHclient provide a VB form design environment of some kind. The reason the "whole bloody DLL vs ActiveX stuff" is important is that you need to get terms right if we're going to communicate meaningfully. I'm starting to realize that I misunderstood a lot of what you've said so far because you're using terms in a rather unconventional and confusing way.

I can't help but think that what you call "relatively minor limitations" are a lot more fundamental than you seem to acknowledge. I mean, this stuff can't just work by magic, and to get this stuff working, Nick would have to hard-code in some kind of well-known interface that everything understands. You can't just get it working without having an interface for things to happen.

For example, if you want to use COM (which seems to be what you want with this 'Advise' stuff) then MUSHclient would have to act as a fully functional COM control server, with all the ups and downs that go with that.

It would help me understand if you didn't go into lengthy discussions about problems, and you instead formulated precisely the requirements of what you're trying to do.

Maybe you could write your own skeletal application that demonstrated some of this in action, so that we could have a proof of concept of how this would actually work?
Australia Forum Administrator #21
Quote:

win = createobject("Some window object")
win.parent = GetFrame
button1 = createobject("VBButton")
button1.parent = win
button1.lable = "Press me!"
...
Advise ("Button1_Click", "OnBut1_Click")

function OnBut1_Click ()
note "Someone clicked me!!"
end function


OK, here is the problem. Your plugin has created an object, which is a button. It isn't clear to me right now where that button is going to go but nevermind that.

Now MUSHclient knows absolutely nothing about this button, as it was created by a script. It doesn't know its window address (handle or whatever), its ID or any identifying details. As far as it is concerned the button doesn't exist.

Now you have called a function Advise passing it two strings: "Button1_Click" and "OnBut1_Click". The second is obviously the script you want called, but how is MUSHclient supposed to translate a "click" message on a window about which it knows nothing, and may possibly never even get, to be something to do with "Button1_Click"? The word button1 is just an identifier local to your script.

I can't begin to get my head around how this might work, sorry.

USA #22
It's a very VBish way of doing things. In VB if your button control is named "Button1" then the click event is "Button1_Click" (or Button1_OnClick, something like that). Basically to get that working you would need control over the windowing components Shadowfyr wants to create, which would mean that you would have to provide the scripting functions, and which basically means you would have to in a sense reimplement a lot of what VB and similar languages provide in terms of a GUI management system.

To clarify, Button1_Click is not really the event, rather, it's the name of the subroutine that is called when the button is clicked. So it's the event in a sense, but it's more than just that because it's truly a function.

Unless Shadowfyr has some tricks up his sleeve that he hasn't revealed yet, I am also completely stumped as to how this is supposed to work without MUSHclient being turned into a full-fledged widget application. (And that seems to go against the stated requirement that the plugin author be free to do whatever s/he wants, because the author would be "stuck" with whatever interface MUSHclient provides.)
Amended on Sat 08 Jul 2006 04:49 AM by David Haley
USA #23
Ok Nick. Here is the thing. From what I have been reading, Mushclient doesn't **need** to know **anything** about the button at all. Seems strange right? But you have to realize that what COM is really doing, under the hood of MFC, where you can't see it, is getting the entry points for "Mushclient's" functions, then passing those pointers to the Advise function in the ActiveX control. Every single button, window, etc. that Mushclient uses is part of the comctl32.dll or other system components, all of which are ActiveX and all of which use Advise and Unadvise to connect their "out bound" functions to the places in your program that need to be called by them, for you to know if someone clicked on something like the "Reload Script" button in the client.

The *only* difference between creating and object "after" you compile the program or "before" is that the compiler can pre-determine the correct function locations in your application, so that it can "Advise" the controls of the correct place to call, without having to figure that out when the program starts every time.

Now, when ever you create something like a callback for plugins or scripts, you are basically replicating what is being done in COM. You are fining the correct "location" in the script to call for that callback, if it exists, storing that someplace for later use, then calling it when your "event" happens, which in this case is something like OnPluginInstall. With COM this works similarly, only instead of the "client" keeping track of the connections, COM does. My example isn't totally accurate, its an approximation. The Advise step would look more like this:

' I assume for consistency MS uses the same IID here, but I can't be sure.

IID_Click = &h(blah)

' This is the "location" of the script entry point for the function, name it what ever you want,
' but I happen to think VB's way of doing it is clear on what you're getting.

ptr = GetRef("OnBut1_Click")

' Tell button1's COM connection point handler that I want the function at "ptr" called when
' someone performs "IID_Click", or in other words, clicks on the button.
' NOTE: This may be button1.IUnknown::Advise. The documentation isn't "real" clear to
' the point I have read so far, since it tends to give examples of how things "work" not how
' you actually bloody use them. :p

button1.Advise (IID_Click, ptr)


Mushclient, assuming it "had" the ability to do this, "still" would *still* have no idea that the window you created "or" the button exists at all, all it would ever see is a request to pass the correct event number, and the script function pointer, to the buttons own event handler, which is simply an internal table that it uses to track a) which events are connected to other applications and b) which functions to call when those events happen.

In fact, the reason it works this way is that, unless you explicitly code the object to not allow it, it doesn't even care how many different things are being informed (having their functions called), as long as they are all valid pointers. You could just as easilly do something like this:

IID_Click = &h(blah)
ptr = GetRef("OnBut1_Click")
ptr2 = GetRef("OnBut1_Fud")
ptr3 = GetRef("OnBut1_Gunk")
ptr4 = GetRef("OnBut1_Ohwhatthehell")
button1.Advise (IID_Click, ptr)
button1.Advise (IID_Click, ptr2)
button1.Advise (IID_Click, ptr3)
button1.Advise (IID_Click, ptr4)

The button would try to call "all" of them when you clicked on it, assuming that its not designed to return a failure when you request more than one connection.

Now, I can see "one" reason why Mushclient may need to know that the window and button exist. Its highly unlikely that reloading a script will "correctly" unload such objects, which means you could have a window with a button on it, which suddenly no long had the function it needed to call to work properly. Click the button and "boom!!". The fix to this "would" require Mushclient to be aware of things. It would need to create the object itself, or at the bare minimum, keep track of it, so that if someone reloads a script/plugin, it can "Unadvise" the objects of the previous pointer, recalculate the correct pointer for the script function then "Advise" the object of that new location. Same with if someone uninstalls a plugin, it would need to Unadvise, then release the objects. But this is a matter is safety, not an absolute necessity. Assuming no one ever reloaded, reinstalled or uninstalled plugins or scripts, the problem would never arise. And I am only "assuming", based on how I understand things working, that the window and button would remain behind if the script was reloaded, instead of being automatically released. I have certainly had some strange crashes when playing with the ActiveX Winamp control, but I haven't "quite" been able to figure out what caused them. It might be, instead, due to the script trying to call an object it thinks still exists, after Winamp has closed and unloaded the control.

Anyway, I hope I have explained better. Its not the application that handles who talks to who and when, its the object itself doing that. All the applications do is tell the object what they should call, in themselves, or in a script they are running, when something happens. Its the ActiveX object that keeps track of who to call and how many there are that need to be called when something happens. The application, except for a case like above which "may" be a problem, or may just be my paranoia, never even needs to know the object in question exists or what it is.
Amended on Sat 08 Jul 2006 06:24 PM by Shadowfyr
Australia Forum Administrator #24
Quote:

Every single button, window, etc. that Mushclient uses is part of the comctl32.dll or other system components, all of which are ActiveX and all of which use Advise and Unadvise to connect their "out bound" functions to the places in your program that need to be called by them, for you to know if someone clicked on something like the "Reload Script" button in the client.


I hate to disillusion you, but most of MUSHclient does not use COM at all. I can prove this to you. One of the first things MUSHclient does (after parsing the command-line parameters) is to check if you used the /wine option, and if not, initialize COM:


// initialise COM

  if (!bWine)
    CoInitialize (NULL);


You can look up this function on the Net, but this is the help that ships with their compiler:


CoInitialize

Initializes the COM library on the current apartment and identifies the concurrency model as single-thread apartment (STA). Applications must initialize the COM library before they can call COM library functions other than CoGetMalloc and memory allocation functions.


Now in case there is confusion, let's check the definition for ActiveX from their help:


ActiveX

A set of technologies that enables software components to interact with one another in a networked environment, regardless of the language in which they were created. ActiveX™ is built on the Component Object Model (COM).


So, the bottom line is, if you run MUSHclient with the /wine option (provided for Linux users) then it doesn't initialize COM, and therefore cannot use ActiveX either.

So, walk over to your PC and run MUSHclient with the /wine option. You won't be able to use VBscript (which uses COM) but 99% of it works - buttons, menus, windows etc.

My point is that MUSHclient does not use COM or ActiveX "underneath" to do everything, and thus there is no "missing link" that I am refusing to provide that will link it to your ActiveX controls.
USA #25
Hmm. Ok. I think there is a secondary way you can get at many of those controls, so I am wrong in that respect. I am not wrong in how COM itself works in this respect. I'm reading the bloody book on it right now, which explains in detail how the COM system was developed, what it does and how it works. I got the comctl32.dll = part of COM detail wrong. So sue me. That doesn't change *why* things won't work when trying to deal with events, or alter the fact that buttons, windows, etc. that "are" accessable as COM objects "do" work the way I am describing.
#26
How do I build a working test.dll file with MUSHclient 4.43 using Visual C++ 2008 Express?

test.c

#ifdef _WIN32
#define LUA_API __declspec(dllexport)
#endif

#pragma comment( lib, "lua5.1.lib" )

#include "lua.h"
#include "lualib.h"
#include "lauxlib.h"
#include "luaconf.h"

#define PI (3.14159265358979323846)

static int miles_to_km (lua_State *L)
  {
  double miles = luaL_checknumber (L, 1);
  double km = miles * 1.609;
  lua_pushnumber (L, km);
  return 1;   /* one result */
  } /* end of miles_to_km */

static int circle_calcs (lua_State *L)
  {
  double radius = luaL_checknumber (L, 1);
  double circumference = radius * 2 * PI;
  double area = PI * radius * radius;
  lua_pushnumber (L, circumference);
  lua_pushnumber (L, area);
  return 2;   /* one result */
  } /* end of miles_to_km */

static const luaL_reg testlib[] = 
{
  {"miles_to_km", miles_to_km},
  {"circle_calcs", circle_calcs},
  {NULL, NULL}
};

/*
** Open test library
*/
LUALIB_API int luaopen_test (lua_State *L)
 {
  luaL_openlib(L, "test", testlib, 0);
  return 1;
 }


files

lua.h      (www.gammon.com.au/files/mushclient/lua5.1_extras/lua5.1_lib.zip)
lualib.h   (www.gammon.com.au/files/mushclient/lua5.1_extras/lua5.1_lib.zip)
lauxlib.h  (www.gammon.com.au/files/mushclient/lua5.1_extras/lua5.1_lib.zip)
luaconf.h  (www.gammon.com.au/files/mushclient/lua5.1_extras/lua5.1_lib.zip)
lua5.1.lib (C:\Program Files\MUSHclient\lua5.1.lib)
lua5.1.dll (C:\Program Files\MUSHclient\lua5.1.dll)


options in VC++

project type: Win32 Project
application type: DLL
added directories for include and library files
renamed test.cpp to test.c
selected "Not Using Precompiled Headers" in test.c property page
all other settings are default


error message in MUSHclient

assert(package.loadlib("test.dll", "luaopen_test"))()

Error number: 0
Event:        Run-time error
Description:  [string "Command line"]:1: The specified procedure could not be found.
 
stack traceback:
	[C]: in function 'assert'
	[string "Command line"]:1: in main chunk
Called by:    Immediate execution


I built a working test.dll file using MinGW (GCC).

gcc -shared -o test.dll test.c -L.\ -llua5.1
gcc -mno-cygwin -shared -o test.dll test.c lua5.1.dll


And I can get the original test.c working with MUSHclient 3.79 using Visual C++ 2008 Express and also using MinGW (GCC).
Amended on Sat 09 Jan 2010 03:26 PM by Morat
Australia Forum Administrator #27
Sounds like _WIN32 is not defined. Try adding:


#define _WIN32


... to the very first line of the .c file, or maybe adding in _WIN32 as a define into the project properties.
#28
MUSHclient gives the same error when #define _WIN32 is at the top.

I read about dependency checkers on the package.loadlib doc page.

http://en.wikipedia.org/wiki/Dependency_Walker

The MinGW test.dll file has an entry point and requires these .dll's:

kernel32.dll
lua5.1.dll
msvcrt.dll
ntdll.dll

The VC++ test.dll file does not have an entry point and requires these .dll's:

kernel32.dll
lua5.1.dll
msvcr90d.dll
msvcrt.dll
ntdll.dll

I guess I'm not settings the correct build options.

Thanks for the response.
Amended on Sat 09 Jan 2010 03:31 PM by Morat
#29
Okay got it working :)

The following code works with MUSHclient 4.43 using Visual C++ 2008 Express and also using MinGW (GCC).

test.c

#pragma comment( lib, "lua5.1.lib" )

#include "lua.h"
#include "lualib.h"
#include "lauxlib.h"
#include "luaconf.h"

#define PI (3.14159265358979323846)

static int miles_to_km (lua_State *L)
  {
  double miles = luaL_checknumber (L, 1);
  double km = miles * 1.609;
  lua_pushnumber (L, km);
  return 1;   /* one result */
  } /* end of miles_to_km */

static int circle_calcs (lua_State *L)
  {
  double radius = luaL_checknumber (L, 1);
  double circumference = radius * 2 * PI;
  double area = PI * radius * radius;
  lua_pushnumber (L, circumference);
  lua_pushnumber (L, area);
  return 2;   /* one result */
  } /* end of miles_to_km */

static const luaL_reg testlib[] = 
{
  {"miles_to_km", miles_to_km},
  {"circle_calcs", circle_calcs},
  {NULL, NULL}
};

/*
** Open test library
*/
LUALIB_API int __declspec(dllexport) luaopen_test (lua_State *L)
 {
  luaL_openlib(L, "test", testlib, 0);
  return 1;
 }
Amended on Sat 09 Jan 2010 05:31 PM by Morat
USA #30
I notice that in the first snippet, you defined LUA_API, but used LUALIB_API on the function instead. Maybe that had something to do with it?
#31
Hi Nick,
I tried this example but somehow i doesn't work.
I use Microsoft VC 2010 to generate the dll, I worked. But when I call it from lua executable(I use lua5.1.4 for windows), it says: nil The specified module could not be found.

Quote"This might be because you didn't put the DLL where it could be found (or the lua.dll or lualib.dll were not found either)."

I already copied the test.dll lua51.dll lua5.1.dll to the same directory as the lua executable.

Is there anything else I missed? Or is there any setting work I should do to tell lua program to find those dlls before I execute the code?

Thanks,
Mingshao
Australia Forum Administrator #32
Do you have a dependency checker with Visual Studio? That usually tells you of related DLLs that might be required (some of them are a bit unexpected).
#33
Hi Nick,
That helps a lot, Thanks, but still, the problem hasn't been solved yet.

I put test.dll into the Dependency Walker, find out all the dll that dependend,(like msvcr100d.dll ntdll.dll,etc). I copied all those dll to the same directory as lua esecutalbe is.

Still, the error messages says:
"nil The specified module could not be found.
init"

And inside the Dependency Walker, in some dll, the "Entry Point" says: "Not Bound". I don't know if that has something to do with this problem.

P.S. Lua Compiler, I use lua5.1.4 for windows, something called SciTE.

Thanks,
Mingshao
Australia Forum Administrator #34
Did you read all of page 1 of this thread? You may need to put the DLL into c:\Windows or something like that.

Or you could try compiling it with Cygwin.
#35
Yes, I did read all of the page 1.

I copied all the dlls and lua executable to C:\windows\System32, and it reported the same problem. Also I copied the test.dll to MUSHclient, it print nil, too. And I want to do some pure in Windows, so I prefer not to try CYgwin.

Now I am thinking maybe the test.dll that I created is not suitable for lua5.1 to call...

#ifdef _WIN32
#define LUA_API __declspec(dllexport)
#endif

#pragma comment( lib, "lua5.1.lib" )
#pragma comment( lib, "lua51.lib" )

#include "lua.h"

#include "lauxlib.h"
#include "lualib.h"


#define PI (3.14159265358979323846)

static int miles_to_km (lua_State *L)
{
double miles = luaL_checknumber (L, 1);
double km = miles * 1.609;
lua_pushnumber (L, km);
return 1; /* one result */
} /* end of miles_to_km */

static int circle_calcs (lua_State *L)
{
double radius = luaL_checknumber (L, 1);
double circumference = radius * 2 * PI;
double area = PI * radius * radius;
lua_pushnumber (L, circumference);
lua_pushnumber (L, area);
return 2; /* one result */
} /* end of miles_to_km */

static const luaL_reg testlib[] =
{
{"miles_to_km", miles_to_km},
{"circle_calcs", circle_calcs},
{NULL, NULL}
};


/*
** Open test library
*/
LUALIB_API int luaopen_test(lua_State *L)
{
luaL_register(L, "test", testlib);
return 1;
}

about dll compiling, I use Microsoft VS2010, I added lua5.1.lib, lua51.lib to Project Properties -> Configuration Properties -> Linker -> Input -> Additional Dependencies.

Do you think is it possible that the problem is in the test.dll? I did make a little change in the codes according the difference between lua5.0 and lua5.1.

Thanks,
Mingshao

P.S. Sorry about this looooooong message
Australia Forum Administrator #36

#pragma comment( lib, "lua5.1.lib" )
#pragma comment( lib, "lua51.lib" )


Make up your mind which one you want to use. I wouldn't use both.

Quote:

And I want to do some pure in Windows, so I prefer not to try Cygwin.


Cygwin can generate DLLs that don't rely upon any other part of Cygwin. I use that to generate the Lua DLL for inclusion in MUSHclient.
Australia Forum Administrator #37
Mingshao Zhang said:

Now I am thinking maybe the test.dll that I created is not suitable for lua5.1 to call...


This is the sort of thing I would expect to see in the dependency walker:



Note that entry point (E symbol) giving the name of the module that you call in Lua. In this case:


assert(package.loadlib("windows_utils.dll", "luaopen_windows_utils"))()


Also note the reference to lua5.1.dll - so this is the DLL you need to have available.
#38
Thanks, Nick.
I found out my problem, when I put my test.dll into the Dependency Walker. There is no entry point.
I will try to fix this
#39
Thanks a lot, Nick

I finally make the example to work, It turns out to be that I always forgot to write the test.def file.

You have been so helpful these days, Thank you so much!!!

Mingshao Zhang