Using the socket library

Posted by Nick Gammon on Sun 28 Nov 2004 12:19 AM — 57 posts, 217,119 views.

Australia Forum Administrator #0
I found on the Lua web site, a "socket" implementation for Lua that lets you do various things - a lot of things - most of which I haven't found a use for yet.

However I'll describe a simple operation - downloading a web page into a variable, to describe the general idea of using Lua extensions.

First I would download this file (or whatever the current version is):


luasocket-2.0-beta2.tar.gz


Unzip that, and inside you will find various things, including some useful lua files in the "lua" subdirectory:


ftp.lua
http.lua
ltn12.lua
mime.lua
smtp.lua
socket.lua
tp.lua
url.lua


In the "etc" directory you also want "lua.lua" which implements the "requirelib" function.


lua.lua


Also, for Windows use, download this file (or whatever the current version is):


luasocket-2.0-beta2-win32.zip


Unzip that, and inside you will find precompiled DLLs for using the socket library:


lua.dll
lua.exe
lua.lua
lualib.dll
luasocket.dll
mime.dll


You get a couple of bonus files there, another copy of lua.lua, and a lua.exe which is a standalone lua executable, along with its support dlls.

The two files you really need from there are luasocket.dll and mime.dll.

Copy all these files to the MUSHclient default directory, whatever that is. As long as they are available to the scripting engine.

I found all the above files at:


http://www.tecgraf.puc-rio.br/~diego/luasocket/


Now we are ready to start. The following script should pull in a web page and display it in the output window:


dofile "lua.lua"	-- installs requirelib
http = require "http"	-- loads http and various dlls
a, b, c = http.request("http://www.somewebsite.com/")
print (a, b, c)  -- page, return code, headers


Of course you only need to do the "dofile" and "require" lines once per session. Once the library is initialised you can pull in a web page in a single line.

From their documentation, these are some of the things you can do:

Quote:

The most used modules implement the SMTP (sending e-mails), HTTP (WWW access) and FTP (uploading and downloading files) client protocols. These provide a very natural and generic interface to the e functionality covered by the protocols. In addition, you will find that the MIME (common encodings), URL (anything you could possible want to do with one) and LTN12 (filters, sinks, sources and pumps) modules can be very handy.

Greece #1
Does this freeze MC while the operation is being performed, like the VBscript ones?
Australia Forum Administrator #2
I think it would have to. The problem with any sort of scripting that interfaces with external modules is, that if the script is asynchronous (ie. it runs independently) at what time does it get resynchronised?

There may be support for it, I'm not sure how it would work.

Perhaps you would start up a session and come back to it later? However I don't see how the script can be doing anything useful if MUSHclient has exited the scripting engine.
Greece #3
True, it can't give you the return value until it finishes... I was just wondering if they had done something else, like call a DLL and have that do it or something. Never mind me :p
USA #4
This is the one annoying factor about scripting. I am sure it is possible to parallel it, but how... If Lua had the equivalant of VBs function then something like this would be possible:

do
a = a + 2
doevents
loop until a = 500

Where 'doevents' basically tells VB to break out of the loop and perform all other tasks before returning to the loop. Scripts do run in parallel in browsers or the like, so I have to wonder if the interface doesn't allow some sort of step through. I.e. Start executing command 1, return to the main program to do stuff, step to the next command and execute, etc. Though how to handle http and other things... I wouldn't even know where to look, but it seems odd that some method does not exist to handle this...
Greece #5
That would still hang MC. MC has to wait for the function to be completed before it is reactivated. This would make the other programs more responsive, but would leave MC frozen for longer.
USA #6
Your are incorrect actually. 'doevents', as least in VB, tells the system to handle 'all' events, including others within the same program, otherwise it would be a useless function, since other applications continue to work even when your program infinite loops anyway (except in cases where you set the program to explicitely behave unfriendly). The problem with http and sockets is a rather bigger issue, since the result does not return from some application until 'they' are done. These dlls are not intended to work in a script. If they where, the code would look like this instead:

dofile "lua.lua";
http = require "http";
a = http.request("http://www.somewebsite.com/");
do 'or whatever Lua uses for this.
  t = http.done
  if (t) {
    b = http.page;
    c = http.header;
    print (a , b);
  }
loop until
print (a, b, c);

Or something like that. This is how you handle talking to IE, you tell it to go to the page, then check to see if it has finished, then do whatever is left when that is true. Socket functions are 'assumed' to be in use by processes that either run parallel to the main application or where it doesn't matter if it hangs the process that called it. You can do the same thing will calls to API in the other languages, and it has the same problem. The only fix for it is to seperate the code that handles it into an external DLL, then basically do this in it:

1. Accept request for socket event.
2. Spawn a new object to handle it.
3. Return a success/waiting event.
3a. When done place data in a buffer for the original event, then kill the spawned object.
4. Accept a 'are you done' request.
5. If done, report yes.
6. Accept requests for the actual data.

Since objects inside a dll can act indepentently of the part that initiated them, at least in the proper threading model, it should 'fix' the issue. The only other option is to make your own implimentations of the protocols that work this way, so nothing ever hangs. But the basic problem is that all of these systems assume either that the calling process can/will wait for the event to finish or that they can respond to events, which is also not usuable in most Mushclient scripting, but is basically the same things as I am talking about about, except that the event handler just waits, instead of explicitly asking it everything finished.

Socket functions are simply not practical within the limitations of mushclient scripting and even using most external aplications/libraries (since they rely on events to return results) are problematic in most of the languages. But even the above alternate method to talk to things requires you hack up the code into multiple sections, since one of then has to use a timer to retest. A 'doevents' type function 'would' fix that issue if such a thing existed in scripting.
Amended on Sun 28 Nov 2004 07:59 PM by Shadowfyr
Australia Forum Administrator #7
Quote:

Start executing command 1, return to the main program to do stuff, step to the next command and execute, etc.


Well, it is now possible!

See this separate thread:

http://www.gammon.com.au/forum/bbshowpost.php?bbsubject_id=4941

In that I demonstrate how you can make a Lua script that yields execution back to MUSHclient, but then resumes at the point at which it stopped, with all internal variables intact.
Amended on Sun 28 Nov 2004 07:52 PM by Nick Gammon
Australia Forum Administrator #8
Searching through the Lua socket library reveals coroutine.yield quite a few times, so it is possible they have allowed for that. I just haven't worked out how yet.
USA #9
Huh.. Lua is sounding more and more like my dream scripting language. lol Now if there is a way to consistently like external events into the script and have them responded to, then it will have everything. This is the one thing really missing from VBScript, JScript, etc. Though I think you can manage it in Python and others. It drove me nuts, since it is a lot easier to, for example, have your music player inform the script that a song changed, than it is to keep asking for the song title over and over to verify that it hasn't.
USA #10
Hmm. Looked at that and.. If you make the function to handle the socket stuff into a coroutine itself, then activate it only as needed, then I think it will execute independant of the calling script anyway. You don't even need to yield anything. I think the coroutines run independant or the need to return a value in the script section that called them, so having that exit would not necessarilly interfer with what the coroutine is doing. In theory anyway... It should work, since your tickle thing does and that shouldn't be possible if having the calling script exit would cause the coroutine to collapse as well.
Amended on Sun 28 Nov 2004 08:25 PM by Shadowfyr
Australia Forum Administrator #11
First, to comment on yielding with sockets, reading the Lua manual a bit more:

http://www.lua.org/pil/9.4.html

This whole section talks about doing threads with reading HTTP. It seems you can set the socket library to non-blocking, and then see if there is more data to be received, and if not, yield. So it can be done.

Quote:

It should work, since your tickle thing does and that shouldn't be possible if having the calling script exit would cause the coroutine to collapse as well.


Why it works is that although the script has ended (ie. the script call has returned to the C++ code in MUSHclient) the script "space" still exists. This is why variables persist from one script call to another (in all scripting languages).

What happens (in all languages, including Lua), is that when you intialise scripting (normally at MUSHclient startup, but also when you reload the script file), a script "engine instance" is created. Into this instance is read the global script file, thus any functions in that are immediately available (eg. to triggers).

Also, any "send to script" in triggers, timers, aliases also use the same script space, so it is possible to share data between them.

When the thread in Lua yields, it is returning execution to MUSHclient, but all its internal data is still in the script engine space.

BTW, each plugin has its own script space, so you could write a plugin the behaved independently from other plugins (as you realise, no doubt).
Amended on Sun 28 Nov 2004 08:39 PM by Nick Gammon
USA #12
Yep. I figured that had to be the case. I found the solution to the event management issue too I think:

events_handler = {}
function events_handler:NewValue(new_value)
  print(new_value)
end
events_obj = luacom.Connect(luacom_obj, events_handler)


Since, as you say, the instance is still up, an event 'should' case the related script to execute, even if there is no active script at the time. This is what drove me nuts about VBScript and JScript. Both support ways to do this, but only with the interfaces specific to the web browser they are running in. The CreateObject functions in them do not correctly expose the complete interface or provide the means to connect to an event, unless it is one created by and exposed indirectly through the interface in the browser. Bloody pain in the rear imho.
Australia Forum Administrator #13
I'm not convinced that will work, however I have added a new feature into 3.56 which should hopefully let you trigger events from outside MUSHclient.

There is now a script function "UdpListen". Basically this lets you set up a listener for incoming UDP packets. Now with suitable programming, a separate application just has to generate a single UDP packet (a pretty trivial task) to cause a script function to fire.
USA #14
Umm.. That is an interesting solution, but kind of pointless. There are already ways, including using callplugin from the external application, to achieve that, if you can code the application yourself. The problem has always been using applications or DLLs that you have not developed yourself, so are unable, or where it is impractical, to redesign the interface to talk to Mushclient indirectly. This is just another indirect solution that doesn't deal with the real problem. You may be correct in that using the events won't work, but if so, another hack which requires extra coding on the part of someone that may not have access to the code isn't going to fix it.

Now, Lua, Python, etc. all have code that reads the library data from ActiveX components and uses that (in the case of lua, through 'connect') to create an event handler. The code and the handler basically do this:

1. Pass handle of object to handler code.
2. Read objects interface table for list of events.
3. Listen for events.
4. When event fires, call the function linked to this event.

Lua's method gets a little different, since it uses <event_handler>:<event_name> to define the function, then calls that when the event happens. I think that this may be because Lua creates event handlers internally for each even at the creation of the object, so when you create one of your own, you are basically overriding the default code (which doesn't do anything). Others simply connect (when they can) to the any function you tell it too, others will simply check for a function called event_<event-name>. So if you had an event called scream, it would automatically assume the script has a function called event_scream, then call that if it exists. Frankly, the methods that link a function of a given name to an event are better, since the function can verify the function exists before hand.

Anyway, point is that real event handler are better than trying to find some way to do an end run around the existing feature, just because you think it won't work. Trying to add an event to a script, even if the event handler had to be in Mushclient, is trivial. It only requires the person writing the script to look up the information for the program or dll they are trying to use. By comparison, trying to glue together a program that sits in between and translates it into callplugin commands or UDP packets is *not* that easy. It requires that they basically re-impliment the entire interface for the program or object using what amounts to digital duct tape.

Frankly imho, this relatively minor annoyance is starting to take on Rube Goldberg proportions.
Australia Forum Administrator #15
Quote:

That is an interesting solution, but kind of pointless.


Well, in conjunction with another project I'm about to announce, I was trying to be helpful. I gather what you are trying to do is capture the "songchange" event from Winamp?

I've done a bit of a search of the web and haven't seen many examples of how people might be doing that in practice.

How does this work exactly? Does Winamp actually post a message into your app's Windows event queue (eg. MUSHclient in this case)? eg. the WM_SONG_CHANGE_EVENT? Or does it call some code somwhere?

What I thought might be useful is that if Winamp can be configured to do "something" at song change time, then that something might be to run a small program that sends the UDP message to MUSHclient.
Greece #16
You can use a Winamp plugin, but the default COM plugin doesn't have events. You could set a timer to refresh every second, but since you'll only be satisfied with an event, that's unacceptable :P
USA #17
I don't use teh default plugin. Someone else made one that has the one event for song changes. This is the one case I have used such an interface, and the inability of it to work right stopped me from designing other things myself that would have used it.

As for how the WinAmp plugin works Nick, it is like any other ActiveX component. Yes, it does pass the events along to the application that created the instance. The problem is that Mushclient 'is not' the application that created it, the script engine was. So it is the script engine that is recieving the events fired by the plugin. Lua is designed to deal with this by instancing a connection point/event handler, which seems to do:

1. Read the objects internal interface layout.
2. Assigns each event found to a function.
3. Creates dummy versions of each function.

Then you overwrite the dummy function with the real one in your script. Since the script remains in the engine at all time, in theory, Lua, and others that support it, should automatically start up and respond to such events when they arrive, since its own event manager is supposed to do that. At least it 'should', though I haven't tried testing it.

Not sure it is would be WM_SONG_CHANGE_EVENT though. The source code for it doesn't show, it just appears to pass the even it recieves itself from Winamp on to the application that instanced it. The source code is at http://mysite.wanadoo-members.co.uk/johnadcock if you are curious and the relevant file is WinampCOMCP.h, where the connection point code is set up for passing on the message. The plugin doesn't need to know what the actually message is and most compilers (and apparently whatever is buried in Lua) don't require the user to know that, just what the general name of the event is. I have no idea how or why it works that way.

However... There may be something in the interfaces for scripts that lets the application using them intercept the messages from such events. But then you would need to read the idispatch interface info and set things up to work. Basically, events in scripts are handled much like the OnPluginLineRecieved and other functions. They are called only if they exist, otherwise they get skipped (or apparently with Lua, they just call an empty function with the right name).
Greece #18
Which plugin is this? I'd like to download it.
USA #19
http://mysite.wanadoo-members.co.uk/johnadcock

Like I said in the last post Poromenos. ;) I did notice a few bugs in it. One or two of the commands didn't seem to work when tried. However, I am sure I posted something on it before that listed the available functions and properties. Ah, here we are:

http://www.gammon.com.au/forum/bbshowpost.php?bbsubject_id=2823

Its the .ToggleRepeat and .ToggleShuffle functions that don't seem to work. Everything else listed works, including the SongChange event, if we could just figure out how to properly respond to it...
Greece #20
Bleh, I should read posts more carefully :P I'll take a look at it :)
Greece #21
ToggleRepeatPlay and ToggleShufflePlay work fine for me...
USA #22
Hmm. I must have misread something myself. I left of the 'Play' at the end of those. :p
Greece #23
I don't think it would be hard to make the plugin send a UDP packet to MC on the event. I don't know if that's what you want, though.
USA #24
Hard no. But imagine if it was something that generates events, but you don't have the code to change it. Nearly every ActiveX program or control does have events that drive it. In the case of WinAmp, it may be possible to modify the plugin to use UDP, but that is a 'very rare' case. Like I said, the only reason I have not even tried to integrate some other things in to Mushclient through script is precisely because in virtually any and all other cases, you would have to build and entire shell around the control or application, just to gain access to the one feature you can't already use through scripting. That is just rediculous and so far beyond what the average script writer could hope to manage that it may as well be impossible. And yes, an 'average' script writer may be willing and able to use ActiveX, just not if they have to create a special wrapper for it to use it right.
Greece #25
Ah, you meant generally. Hmm, can't MC act as the wrapper? Maybe there could be a OnCOMEvent function or something.
Australia Forum Administrator #26
I know I've asked this before, but I just don't see how an external COM object can just kick into life a script whose engine doesn't happen to be running at the time.
Greece #27
Well, MC would ignore the event if the engine wasn't running. (Assuming acting as a wrapper is possible, I haven't worked much with COM so I don't know).
USA #28
Well Nick.. I am not clear what you even mean by that. For the script to remember its own internal variable states it must remain loaded in the engine, even if the engine itself it technically idle. However, I would have to guess that, for event managers to work at all, the engine itself must have code that continues to run, even when the script itself is suspended. Amoung that code would be an event handler, which would capture events from connected objects. If the event handler recognizes an event that is not part of the engine itself, it would check it against objects that it has connection point for and wake-up the script long enough to execute the appropriate code, then suspend it again when done. If something like this was not in fact possible, then there would be absolutely no point in even providing the ability to create such a connection point for an ActiveX object. Events are designed to push program execution, even if there is no 'loop' in the application/script that is in continual operation, aside from a basic one that just makes sure the program doesn't exit until all objects are released.

Guess I will have to actually download the latest version and try some stuff. Been busy with other things, so hadn't gotten around to actually experimenting yet. However, your assumptions that some things can't or won't work flys in the face of the whole intent behind having such features in a script language in the first place.
Amended on Sun 05 Dec 2004 09:45 PM by Shadowfyr
USA #29
Shadowfyr, if Nick says it can't work in MC, chances are that he's right. :-) He did after all design it and he does know what his event handling looks like.
Quote:
Events are designed to push program execution, even if there is no 'loop' in the application/script that is in continual operation, aside from a basic one that just makes sure the program doesn't exit until all objects are released.
Wrong. You need a 'basic loop' that is in continual operation checking for messages.

Besides, there's an awfully big difference between something being possible and something not taking hours and hours of work to do to satisfy an extraordinarily small audience (as far as I can tell, you.)
USA #30
I beg to differ Ksilyan. Yes, Nick may have developed the event management for Mushclient, but scripts could not work as stand alone systems, unless the engine also had its own event manager. I am pretty sure they use 'that' handler for object events, since Mushclient *is not* the program creating the objects, the script engine is. It doesn't make a lot of sense for it to rely on the host application, which is apparently not even aware of such objects or able to communicate with them.
Australia Forum Administrator #31
Well, not really. I can't imagine personally how an external event (like a song change) can kick into action a piece of code that is pseudo code that needs to be executed by a script engine, which isn't running at that point. Maybe it can be done, I would be interested to hear how.

However I do know that at present your scripts don't "just respond" to external events, MUSHclient calls them in a predetermined way.

It goes like this:

  • MUSHclient is in its main Windows event loop
  • An event happens. Depending on what it is, it goes to the appropriate handler. eg. screen redraw causes it to repaint the screen (perhaps you had another window on top of it?).
  • A comms event (incoming TCP/IP data) eventually happens
  • It goes into its "process incoming text" routine
  • It handles ANSI codes, MXP etc.
  • Eventually a newline arrives
  • It goes into the "process the previously-arrived line" routine
  • This looks up triggers for that world (if active) and processes each one
  • For "send to script" or a trigger that needs to call a script it:
  • Activates the script engine, telling it to process either the named script event (ie. the trigger script) or simply sends the entire send box to the the script engine (as "send to script")
  • Now the script engine is active and does its thing, while it is doing it, MUSHclient waits for the script call to return.
  • When the script ends, MUSHclient goes on to do different scripts, then "omit from output", send replies etc.
  • MUSHclient processes other triggers
  • It goes back to its main loop


So you can see that trigger events don't just call your script without a considerable amount of intervention. Similarly for other scripted events, each happens at a well-define point in the execution process (eg. incoming chat message, incoming packet, world open, world close).

Now if you say to me that Winamp sends a Windows event (eg, the WM_XYZ_EVENT) then I can put in a check, that if that event (or any event you might define) happens it can call your script. However I haven't seen any suggestion so far that Winamp actually does put events into the event queue.
Amended on Mon 06 Dec 2004 06:46 PM by Nick Gammon
USA #32
But Nick. My point is it *can't* send events to an application that is not aware of the control. The script engine itself does know about it, but as far as Mushclient is concerned such a control does not even exist. How can it recieve the event? However, for the script engine to take advantage of the ability to handle events through a connection point, there must be something 'in the engine itself' that does it. You have to realize that even though you are dealing with dlls, not an exe, and even though there is no 'window', the script engine is still an application and it does have its own internal event management. The only real question is if Mushclient's suspending the script execution also suspends the event manager in the script.

Now I may be wrong... It is possible that since you are running the script engine as a dll in Mushclient, that all its events are first passed through Mushclient, before being passed on to the engine, but that just means you can intercept them. It does not alter the fact that Lua having connection points would be a totally worthless feature, if it didn't also have a means to capture the events that arrive through that connection point. What would be the point of even allowing a script writter to make one if it didn't allow the engine to wake up the script and execute code in response to one? Answer: there wouldn't be a reason, since the 'only' things such connection points are used for is handling events. Properties, function calls, etc. are all exposed and available from the moment you create the control. The only thing the last 6 lines in the following code can or would do:

winampCOM = luacom.CreateObject("WinampCOM.Application")
events_handler = {}
function events_handler:SongChange()
  print "Song changed to " + winampCOM.CurrentSongTitle
end
events_obj = luacom.Connect(winampCOM, events_handler)

is set up a way to deal with the event fired by the ActiveX control.

The explaination for it is here:

http://216.239.57.104/search?q=cache:c-WDyJapmTMJ:www.tecgraf.puc-rio.br/~rcerq/luacom/pub/1.1/luacom-1.1-doc.pdf+lua+luacom.connect&hl=en

Basically, in the above. The first line provides access to everything except the events. The 2nd - 5th lines create a function in a table, so that is one called SongChange is detected, it executes that code. The last line 'connects' the event stream for the object in the first line to the table you provided, with the event functions in it. Now, you may not get how or why it could work, but it must somehow, or why provide the commands to do it?? The only real question is if I got the event name right and what if any effect Mushclient's suspending the script has on this feature. We just don't know. And again, I have no clue if Mushclient ever even sees the event in its own handler or if maybe the dll is treated as a 'sub-window' and gets automatically passed any events the client itself doesn't handle. I don't know how it works exactly with scripts.
USA #33
Perhaps you could whip up a Windows application written in C++ and uses scripting engines that demonstrates this, Shadowfyr. Obviously Nick doesn't think it's possible with the model that he has, I would tend to agree with his judgment for a variety of reasons, so perhaps you could show us old dogs how it's done. :-)
USA #34
He may be right. I am just describing my understanding of how event management works and questioning why, if the engine can't do it, they even bothered providing a way 'for' it to do it. I don't think we need a seperate C++ application to test it. Even if I had a clue how to manage writing one. It either works as the Lua documentation and everything I have read about events says, or Nick is right and Mushclient is a special case where it won't. And that opens up a whole new can of worms. In any case, Nick would be more qualified to capture and examine what Mushclient does or doesn't recieve through it own even stream than I am. If the event does pass through it first, then it will appear as a packet in that stream. The problem is, even if it does, does Nick know how the script engine works well enough to say that such events don't end up handed off to that as well? I don't know any of it well enough to say either, but I tend to wonder when the people that develop a script engine give specific examples of how to do it. The question has to be asked, "If other programs that use this engine really do treat it in a significantly different way than Mushclient does, how and why does that make it not work in this case?" Do we even know that Mushclient uses it in that significantly different a fashion so as to prevent it?
USA #35
Could you show very specific examples in the Lua documentation of what you're talking about, and sample code that I suppose they provide to get it working in C++? If you've done so already I must have missed it. I didn't see the Winamp example in the reference manual you provided.

It also seems that this is about creating COM objects that Lua will use, and not hooking into already running, external objects (e.g. the Winamp application.) There's quite a significant difference between the two.
See page 14 of the reference manual:
http://216.239.57.104/search?q=cache:c-WDyJapmTMJ:www.tecgraf.puc-rio.br/~rcerq/luacom/pub/1.1/luacom-1.1-doc.pdf+lua+luacom.connect&hl=en
Australia Forum Administrator #36
Quote:

... the script engine is still an application and it does have its own internal event management ...


No, it isn't. It is a piece of code sitting inside MUSHclient, in much the same way as the spell checker. It has no life of its own, it doesn't get events, it is "dead" until it is called into life, say by a trigger firing.
USA #37
Hmm. Servers me right for not looking closely at things.. Seems LuaCOM is a seperate modual:

http://www.tecgraf.puc-rio.br/~rcerq/luacom/#Release12

So the existing engine doesn't have support for COM at all. Including even 'basic' allowances for using ActiveX objects of any kind. The download link doesn't seem to work for me though. Basically, it is an addon library that adds the support. Wondered why the attempt I was trying wouldn't work at all.... :(

As for the document I showed. The examples in it are for Microsoft's calender gadget and some other basic things. The code I gave in my post was an adaptation, based on that. Obviously they are not going to have the same example.

Anyway, sorry for the confusion. Seems that you need an aditional library to support it at all. I was assuming that Lua was like Python and had such things built in as a more or less basic part of the language.
Australia Forum Administrator #38
Lua is designed to be cross-platform (unlike, say, VBscript).

Its basic implementation has very little operating system-dependant stuff.

If I had added COM as part of the basic Lua package then I would be back to the problem I already have with the other script languages, you can't use them very reliably on Linux because COM isn't very reliable there.

However there is nothing stopping you from adding the LuaCOM package to your own copy (if you can get the download to work).
USA #39
Not sure how exactly to manage that. The only examples seem to show Lua and LuaCOM being started through C++ in another application or just using the commands. Nothing about how to get Lua itself to use it.

As for the cross platform issue. Python is also cross platform, but it 'does' let you use COM, when on a system that has it. On others it might use CORBA, or it might use the std-in/std-out model, etc. If all you want to do is run a script, you don't need to worry about things like COM support, but the stuff people want to do goes way beyond that. If you did design a Linux version, then yes we would have to adapt, just like anyone confronted with the fact that inter-process/application communications do not work the same between platforms. But then I am not going to be using WinAmp on Linux and I would have to rewrite my Fireworks gadget, or anything else, for that platform anyway, at which time I would have to adopt a different method. However, right now, we have one that does work, is available through most of the script types and is only missing one thing to make it completely bi-directional, as originally intended. I think you can understand my endless frustration with this issue. ;)
USA #40
Quote:
As for the document I showed. The examples in it are for Microsoft's calender gadget and some other basic things. The code I gave in my post was an adaptation, based on that. Obviously they are not going to have the same example.
My point being that the examples were accomplishing very different things than what your "adaptation" was trying to accomplish.

For that matter, I'm still not entirely sure you're clear on what you're trying to accomplish or how to go about doing it; you simply assume that there must be a way since you want there to be one. :-) You should start by making this work in your own example application (that, yes, you'd have to write yourself, in C++) and then, if you get it working there, poke Nick, prove him (and me) wrong, and show us how it's done. :) To be honest I'd be very curious to see it working in the same conditions under which MUSHclient is running...
USA #41
How precisely is mine different?? Here is another example, from http://www.promixis.com/php/drops/iTunes%20Interface%20v0-1.04

...

function iTunes.Connect ()

  if iTunes.Connected then
    return iTunes.Version
  end

  if not luacom then
    UnloadLuaCom ()
  end

  LoadLuaCom ()

  iTunes.com = luacom.CreateObject("iTunes.Application")

  if not iTunes.com then
    return nil, "iTunes: unable to connect"
  end

  iTunes.Connection = luacom.Connect(iTunes.com,iTunesEvent)
  if not iTunes.Connection then
    return nil, "iTunes: unable to link events"
  end
  iTunes.Version =iTunes.com.Version
  iTunes.Connected = 1
  return iTunes.Version
end

...

iTunesEvent = {}
function iTunesEvent:OnPlayerPlayingTrackChangedEvent (iTrack)
  iTunes.CallEventHandlers (3)
end

function iTunesEvent:OnPlayerStopEvent(iTrack)
  iTunes.CallEventHandlers (2)
end

function iTunesEvent:OnPlayerPlayEvent  (iTrack)
  iTunes.CallEventHandlers (1)
end

function iTunesEvent:OnDatabaseChangedEvent (deletedObjectIDs, changedObjectIDs)
  iTunes.CallEventHandlers (4)
end

function iTunesEvent:OnCOMCallsDisabledEvent  (reason)
  if reason == 2 then -- itunes is quitting
    iTunes.Disconnect ()
    iTunes.CallEventHandlers (6)
    return
  end
  iTunes.CallEventHandlers (6)
  iTunes.Disabled = reason
end

function iTunesEvent:OnCOMCallsEnabled  ()
  iTunes.Disabled = nil
  iTunes.CallEventHandlers (7)
end

function iTunesEvent:OnQuittingEvent  ()
  iTunes.Disconnect () -- this routine calls the eventhandlers
end
...

This is 'identical' to what I am attempting, except that it uses the iToons player, instead of WinAMP. If not, then how does it differ?

The steps above are exactly the same as my example. Create an instance of the control (in the case if WinampCOM, this also opens the player), then create a connection point, which links the event handling functions to the event stream of the object. What do you think I have been trying to do that is not the same?
Australia Forum Administrator #42
Well, I'm not sure what luacom offers, but does this application do anything else, except wait for tunes to change?
USA #43
Like I said, Shadowfyr, I'll be convinced the day you come up with sample code in C++ that gets this to work and, as Nick said, does something other than sitting around waiting for iTunes. That is, a full-blown application (say, similar to MUSHclient), that oh-just-so-happens to have this little scripting component that can listen to external events.

Note that the sample script you gave works in an application caleld 'Girder', which claims to be a "Windows automation tool". Chances are, the whole point of the program is to sit around waiting for this kind of event - it is not at all like MUSHclient.
USA #44
Well Nick. LuaCOM is a "COM" library for Lua. It is what allows you to use ActiveX at all. My Fireworks gadget, the mapper for a BattleTech based mud and virtually every other example of using an external application from insides a script all use ActiveX. If you do a google search on 'ActiveX Lua', the **only** pages that list are either the LuaCOM ones or adaptations of LuaCOM as plugins for other applications. Lua itself cannot do it natively, even though every other one, including I suspect PHP, all provide at least basic support for it. So yes, it does do more, since it is more or less 100% necessary to even connect to and control 90% of all the applications currently written for windows.

And Ksilyan.. Yes, Girder is designed to "control programs like WinAmp and Windows Media Player using infrared and wireless remote controls so you can sit back and enjoy your digital library from your couch." I am not sure what if anything the difference technically is between that and driving things from a text stream. This applications seems to have to respond to a wide number of different devices and while the scripts may define what to do about the signals recieved, the major portion of the application likely involved calling several different scripts/plugins, depending on 'which' remote or device signals it. Sounds a lot like triggers firing and calling scripts, but heh, what do I know. LuaCOM just happens to be one of the things installed to extend functionality. Apparently, it is used to drive a LCD Master component, which displays information about what one or more of the devices being controlled is doing.

In any case, I am not at all sure what the heck you seem to want Ksilyan. I don't know C++ well enough to even try writing an example and no examples I have given, including one that uses C++, is good enough for you. You can't seem to get past the potential delusion that Mushclient is so horribly different that it isn't even worth investigation by someone that actually can code something to test it. For now, Lua is useless to me, even for what I have already been using other languages to do.
USA #45
Ok.. Help Nick.. I finally managed to download the LuaCOM and unzipped the needed files (key is don't use Getright, apparently the site doesn't like download managers):

luacom.lua
lua-4.0.dll
lua-5.0.dll
lualib-4.0.dll
lualib-5.0.dll
luacom-lua4-1.2.dll
luacom-lua5-1.2.dll

Into the Mushclient directory. However... When I try to use 'require ("luacom")', I get:

[string "program"]:1: could not load package `luacom' from path `?;?.lua'

When are the people that develop for Windows going to get that environment variables are DOS and you can't expect to have one set to the path you need? Sigh.. Any ideas what to do about this? Apparently using "C:\progra~1\mushclient\luacom.lua" eats the '\', while using '/' keeps them (love Linux to Windows ports...), but still gives the same error. Kind of hard to use external .lua files if it won't let you import them. :(
Australia Forum Administrator #46
Hey, that was fun! First time I've got luacom working.

First, if anyone is looking for it, try here:


http://www.tecgraf.puc-rio.br/~rcerq/luacom/


Now comes the tricky bit <grin>.

In that archive you will find a heap of files, however you only need 2. In the bin subdirectory, copy these 2 files to your MUSHclient installation folder (ie. the one where mushclient.exe is) ...

luacom-lua5-1.2.dll
luacom.lua

Now, and this is the tricky bit, if you open a command window and type:

depends luacom-lua5-1.2.dll

... you will see that it is missing two DLLs:

LUA-5.0.DLL
LUALIB-5.0.DLL

Now that I did was to copy the existing lua.dll and lualib.dll which is supplied with MUSHclient to make a separate file with those names. That way I have the identical copy of Lua in luacom as in MUSHclient. However I note that they were also supplied with the luacom download so you could probably copy those too.

Now, typing:

require "luacom"

works. What this actually ends up doing is the equivalent of:

dofile "luacom.lua"

So, if the search path is confusing then you could do that instead. However it seems to work for me with the default search path of the current directory, provided that is where you put the files.


Quote:

When are the people that develop for Windows going to get that environment variables are DOS and you can't expect to have one set to the path you need?


The documentation for require states that it gets its path from the global variable LUA_PATH, as follows:

Quote:

To determine its path, require first checks the global variable LUA_PATH. If the value of LUA_PATH is a string, that string is the path. Otherwise, require checks the environment variable LUA_PATH. Finally, if both checks fail, require uses a fixed path (typically "?;?.lua", although it is easy to change that when you compile Lua).


Thus if you want to change the path you could have done this:


LUA_PATH = "blah/?;blah/?.lua"
require "foo"


What that does is substitute "foo" for the question mark, and then try to open:


blah/foo
blah/foo.lua





Having read the luacom manual (a bit) I have to say that it looks pretty cool. Using that, you can use COM objects from Lua, so (on Windows at least) you can go crazy and use COM *and* Lua as well.
Amended on Tue 07 Dec 2004 07:00 PM by Nick Gammon
Australia Forum Administrator #47
Having re-read your post, it seems you had all the right files. Anyway, try the depends thing, to check the DLL has all its dependencies. I wasted quite a bit of time before with the loadlib function telling me a DLL wasn't there, when it was, when the problem really was that a required DLL (a DLL the DLL wanted) was not there.
USA #48
Ack.. I am getting too:

luacom.lua:2: attempt to call global `loadlib' (a nil value)

Sigh.. Wonder what is wrong this time. I can't seem to win. lol

And 'depends.exe' appears to be something that is part of your Visual C++ tools or something, not (as usual) something from the OS itself. :( Also apparently doesn't come with any compiler, etc. that I have installed. Sigh... I am sure all the files are there though. This is annoying, but at least it was closer than yesterday.

Ah, found a site to get the utility:

http://www.dependencywalker.com/

--edit--
Well, this is quite annoying. All the right files are there according to the dependency checkers list, but it isn't bloody working... Still, at least I got a nice utility out of it.
Amended on Tue 07 Dec 2004 07:46 PM by Shadowfyr
Australia Forum Administrator #49
Quote:

... 'depends.exe' appears to be something that is part of your Visual C++ tools or something, ...


Yes, I thought it might be. :)

Read this:


http://www.gammon.com.au/forum/bbshowpost.php?bbsubject_id=4910


The default for MUSHclient is to sandbox Lua somewhat. Go to Global Preferences -> Lua. You should see something like this:



--[[

-- Lua initialization 

-- Default initialization supplied with version 3.52.

Use this to create a "sandbox" for safe execution of non-trusted scripts.

If you only run your own scripts or plugins then you may not need this.

The code in this area is executed after each Lua script space is created
but before any of your scripts are done. This can be used to initialise things
(eg. load DLLs, load files, set up variables) or to disable things as shown below.

By setting a function name to nil you effectively make it unavailable.

You can remove some functions from a library rather than all of them, eg.

  os.execute = nil  --> no operating system calls
  os.remove = nil   --> no deleting files
  os.rename = nil   --> no renaming files


This script will automatically be replaced if you completely delete it from
the Global Preferences, and restart MUSHclient. To avoid this, leave a comment
in (if you don't want any further action taken).

--]]

-- Example sandbox --

function MakeSandbox ()

  loadlib = nil   -- no loading DLLs
  io = nil        -- no disk IO
  os = nil        -- no operating system calls

-- These are probably OK, if the above are disallowed --

--  require = nil   -- no loading files
--  dofile = nil    -- no external scripts
--  loadfile = nil  -- ditto

end

-- default is to sandbox everything --

-- MakeSandbox ()  --> remove this line to remove protection

--[[

-- You can limit the behaviour to specific worlds, or specific plugins
-- by doing something like this:

if  GetWorldID () ~= "05becae638cf8ce8ee2baf08"  -- trusted world 1
and GetWorldID () ~= "76cbff65aecf520f42c0df35"  -- trusted world 2
then
  MakeSandbox ()
end

if GetPluginID () ~= "" then  -- don't trust plugins
  MakeSandbox ()
end

--]]



I got the same thing, I forgot to mention. I commented out the line in bold and it worked. Then reload the script file to re-instantiate the script engine.

Or, you could be more sophisticated and do what is suggested lower down, and only make the sandbox for certain worlds.

One of the sandboxed lines is:


loadlib = nil -- no loading DLLs


That will be your problem. :)
Amended on Tue 07 Dec 2004 08:05 PM by Nick Gammon
Australia Forum Administrator #50
Having done all that (copied the DLLs, fixed the sandbox) then this is a quick test:


dofile "luacom.lua"

excel = luacom.CreateObject("Excel.Application")
excel.Visible = true


That worked for me, Excel popped up.
USA #51
Yep. That would tend to explain it. lol

Couldn't you, to be safer, set the sandbox function to load luacom, then nil the function? That way you still supply access to it, but disable the ability to load more. Though... Maybe not, the Luacom dlls may use loadlib themselves in some fashion for all I know, or the functions that access them. But might be worth a try.
Amended on Tue 07 Dec 2004 08:37 PM by Shadowfyr
Australia Forum Administrator #52
Yes, you could do this in the global Lua initialisation:


require "luacom"
MakeSandbox () 


That loads the loacom module and then disables the functions.

However it might be a bit of an illusory gain, as you can now use COM (eg. File Scripting Objects) to do quite a bit of harm.
USA #53
Quote:
I am not sure what if anything the difference technically is between that and driving things from a text stream.
My point, Shadowfyr, is that as you state yourself you really have no idea what you're talking about - you don't know C++ and you've never written Windows applications with it. The main reason I keep asking you to show some C++ code is that I would like you to understand that it's not as easy as you seem to think it is. It's fine if you think it's possible. And if you want to show that it is, why don't you go make a program that does it? Continually insisting that things should be as you want them to be without giving any indication that it's possible in code is esentially hot air. If you don't know anything about it, by your own admission, then why are you trying to tell me that Girder and MUSHclient are the same kind of program? Your silly comparison of Girder's input receiving to MUSHclient's processing of text and triggers is just that, silly. :-)
Amended on Tue 07 Dec 2004 10:16 PM by David Haley
USA #54
Sigh.. Ksilyan.. I have been programming for nearly 25 years or so. I don't happen to have coded in C++, only because I don't have a compiler, I hate the free ones that make what is already more complicated (coding without some auto-code generation for objects) even more complicated and I don't have the cash to buy one. It is irrelivant. Languages are languages, they may each have some quirks, but they all conform to programming logic, which I tend to be good enough at that I can think myself through most of an application before I ever write a single line of the code to make it work. C++ just makes you waste a lot of extra time just setting every bloody thing up to get to the point where you can do what you wanted in the first place. Can't imagine why I wouldn't want to spend a week to get to that point, just to prove mine.... I have tried to code in C++, I just got frustrated at how much extra junk you have to do to make anything work and the lack of any good tools to help design with GCC and other free versions.

But it is all irrelevant. The only *real* question is "What happens in an application where the script runs in the same thread as the application and you want to use Luacom?" And you are right, for all I know Girder runs Lua in a seperate thread, so it never has to actually suspend the script engine between uses. But trying to run more than one script at a time, each in its own thread, is probably going to get complicated. I can't imagine Girder's developer thinking that would be a briliant idea. But who knows, I may be completely wrong about just how big of a mess running dozens of scripts in dozens of seperate threads would really be... I would tend to suspects that most people would try to keep in simpler than that though and employ the same model Mushclient uses, "suspend scripts when not executing them".

But it is irrelevant what I do or don't know about how it works. Nick managed to figure out how to get Luacom to work, so now it is just a matter of testing it to see if event handling works as loaded through the script itself. If not, then it is back to the drawing board. It may end up turning out that the only way to do it would be to add a Win32 specific module to Mushclient, which could be gutted for other systems, that duplicated what Luacom does for events, but is usable in 'all' scripts, not just lua. Heck, we now have some place to get a 'good' source code example showing how to make it work, which is by itself more than I have managed to find in an entire year or searching through google. Lot of general examples, just nothing quite so specific to 'just' making a stand alone event manager, which you can use as a dll, until now.
Amended on Wed 08 Dec 2004 04:11 PM by Shadowfyr
Australia Forum Administrator #55
Quote:

Nick managed to figure out how to get Luacom to work ...


That's right. Looking on the bright side, even if we don't get the Winamp thing to work, our knowledge has increased in other directions. :)
#56
You guys seem really amped up on this lua.

I've been scripting in VB since April 2001... And this Lua sounds pretty amazing, doing everything that I've wanted, pausing, frational second triggers and what not... Sounds like it's going to completely change the way scripting is done... I'm looking forward to it.