New feature in version 4.55 - extended CallPlugin syntax

Posted by Nick Gammon on Wed 28 Jul 2010 04:47 AM — 36 posts, 122,035 views.

Nick Gammon Australia Forum Administrator Wed 28 Jul 2010 04:47 AM #0

Version 4.55 of MUSHclient offers an extended version of the CallPlugin syntax, for Lua-to-Lua calls only.

First, some background ...

Recently MUSHclient plugins have become more sophisticated, as authors have been implementing miniwindows, telnet negotiation, mappers, and caching. From time to time, plugin authors want to share information from one plugin to another.

Up until now, there have been two main methods of doing this:

BroadcastPlugin
CallPlugin

Both of these let plugins pass information from one to another. In the case of BroadcastPlugin a "broadcast" is made, which all plugins can pick up. In the case of CallPlugin, a specific plugin is targeted.

However neither call allows information to be returned to the caller, which makes them less useful for situations like "give me the current mob's name", or "look up a cached item on the database and give me its value".

Up until now, this has been achieved reasonably well by letting the called plugin set a variable, which the calling plugin can query, however this is a bit cumbersome. Twisol has developed a PPI module (Plugin-to-Plugin Interface) but even that suffers a bit from being a bit unwieldy, in that it has to still set plugin variables, albeit in the background.

The old CallPlugin syntax looked like this:


result_code = CallPlugin (plugin_id, function_name, argument)

(Where result_code was 0 for success, and the argument was a single string value).

For example:


result_code = CallPlugin ("80cc18937a2aca27079567f0", "LogIt", "Data to be logged")

The new syntax (which is backwards compatible with the old) looks like this:


result_code [ , val1, val2, val3 ... ] = CallPlugin (plugin_id, function_name, arg1 [, arg2, arg3, arg4 ... ] )

For example:


rc, a, b, c, d = CallPlugin ("80cc18937a2aca27079567f0", "show_message", "red", "green", "message", 42)

An example of the implementation of show_message in the target plugin would be:


function show_message (fore_colour, back_colour, message, count)

  for i = 1, count do
    ColourNote (fore_colour, back_colour, message)
  end -- for

  return 1, 2, 3, 4
end -- function show_message

The calling plugin would call it like this:


rc, a, b, c, d = CallPlugin ("80cc18937a2aca27079567f0", "show_message", "red", "green", "message", 42)
print ("rc =", rc) --> rc = 0
print (a, b, c, d) --> 1 2 3 4

The backwards compatibility exists because the first result from CallPlugin is still the return code, which is still 0 in the case of no error.

If the return code is in fact 0 (otherwise defined as error_code.eOK) then any values returned by the called function are then returned as additional return values (which Lua lets you do, unlike most languages). Also the case of sending a single string argument (which the old CallPlugin did) is now a special case of sending multiple arguments.

If the return code is not zero, then it will be one of the return codes defined for CallPlugin. In addition to that, a second value will be returned which is a string explanation of the error.

For example:


rc, a, b, c, d = CallPlugin ("80cc18937a2aca27079567f0", "foo", "bar")
print ("rc =", rc)  --> rc = 30036
print (a, b, c, d)  --> No function 'foo' in plugin 'Test_Plugin' (80cc18937a2aca27079567f0) nil nil nil

This lets you find out in more detail why a CallPlugin call failed.

In order for all this to work:

Both the caller and called plugin must be written in Lua (the caller does not have to be a plugin)
You can pass between zero and any number of arguments to the requested function
The passed arguments can be one of the following types only:
- nil
- boolean
- number
- string
In particular, the following types are not supported and will result in an error code being returned:
- function
- userdata
- thread
- table
Functions are specific to a Lua script space (for example, they may be implemented in a DLL) and are not portable across script spaces.

Userdata is likely to represent non-portable data, such as open files.

Threads (coroutines) will not be portable.

Tables are not only likely to be deep (eg, the entire _G table), plus are likely to contain functions, userdata and threads.

If you wish to pass a table (eg. a player's inventory) then this can be accomplished by using the serialize library to convert a table into a string.

It is permissible to pass no arguments at all. eg.


CallPlugin ("80cc18937a2aca27079567f0", "my_func")

It is possible for the caller and callee to be in the same plugin (however there is not much point to doing that). If you do that, the restrictions on the types of data that may be passed and returned do not apply.

The return code(s) from CallPlugin are:

On failure:

A status code, as before, which will be non-zero
A string indicating the reason in human-readable form
In the case of a runtime error in the called function, a second string indicating the nature of the error

On success:

A status code, as before, which will be the number zero
Zero or more values as returned by the called function.
The returned values can be one of the following types only:
- nil
- boolean
- number
- string
Thus it would be an error for the called function to return (for example) string.match as that is a function.

The reason for this is the same reason as the restrictions on the parameter passing - the parameters and returned values have to be copied from one Lua script space to another, and only certain data types lend themselves to such a copy operation.

Because of the backwards compatibility, no changes should be required to existing plugins or code. Also if you do not use Lua, then you can continue to use existing methods, such as setting plugin variables, or using world.Execute to cause aliases to fire in other plugins.

Twisol USA Wed 28 Jul 2010 05:17 AM #1

Excellent! I believe PPI still has a place as a plugin-watcher, though. You can't use CallPlugin until the plugin you want to communicate with is available. And CallPlugin is still a little un-pretty just by virtue of passing the plugin's ID every time.

I also can't help but disagree with this:

Nick Gammon said:
*It is possible for the caller and callee to be in the same plugin (however there is not much point to doing that). If you do that, the restrictions on the types of data that may be passed and returned do not apply.

See bolded. I don't like the inconsistency you're introducing. It may make sense looking at it from the implementation of the function, but to user code it can be confusing. You expect it to have a specific contract, and the function even looks like you're calling out to another plugin. So if you break a plugin into two plugins and it relied on this behavior, you have a problem.

Not to mention anyone using CallPlugin for this kind of thing can only be new, and we should try even harder not to encourage this kind of thing. After all:

CallPlugin("f59a5eb513655f255efc07fe", "foofunc", {1, 2, 3, 4})
-- v.s.
foofunc({1, 2, 3, 4})

It's easier and clearer to use the second form.

I'll pull down the changes and see about refitting PPI. Thanks for adding this support!

WillFa USA Wed 28 Jul 2010 05:26 AM #2

In theory would this work if we wanted the result list packed into a table?



function CallPlugin (...)
  local function pack(...)
   return {...}
  end

  local t = pack(world.CallPlugin(...))

  if table.remove(t, 1) == 0 then
     return t 
  else
 -- error () -- handler...
  end
end




ret = CallPlugin("0000aaaabbbbcccc", "foofunc", 1, 1, 3, 4, 7)

Twisol USA Wed 28 Jul 2010 05:36 AM #3

I don't see anything wrong with it, so my guess is yes.

Nick Gammon Australia Forum Administrator Wed 28 Jul 2010 05:46 AM #4

It doesn't have to be that complex:



t = { CallPlugin ("80cc18937a2aca27079567f0", "show_message", "red", "green", "message", 42) }

tprint (t)

If a function returns multiple values and is put inside a table constructor, its values get turned into table items.

In other words, you don't need the helper function (unless you want to delete item 1 from the table, which you can do now anyway).


t = { CallPlugin ("80cc18937a2aca27079567f0", "show_message", "red", "green", "message", 42) }

if table.remove (t, 1) == error_code.eOK then
  -- t contains results
else
  error (t [1] )
end

Twisol USA Wed 28 Jul 2010 05:49 AM #5

Or:

t = {assert(CallPlugin ("80cc18937a2aca27079567f0", "show_message", "red", "green", "message", 42))}

Ugly enough to warrant a wrapper, perhaps:

do
  local orig = CallPlugin
  function CallPlugin(...)
    return {assert(orig(...))}
  end
end

t = CallPlugin("80cc18937a2aca27079567f0", "show_message", "red", "green", "message", 42))

Nick Gammon Australia Forum Administrator Wed 28 Jul 2010 05:52 AM #6

Anyone that does this:


CallPlugin (GetPluginID (), "foofunc", {1, 2, 3, 4})

Rather than:


foofunc({1, 2, 3, 4})

... deserves all the pain he gets, honestly.

And why stop there?


CallPlugin (GetPluginID (), "CallPlugin", GetPluginID (), "foofunc", {1, 2, 3, 4})

... and so on.

The fact is, that with only one Lua script space, I can't just copy from L1 to L2 (as I do with two script spaces) because then you get all the parameters twice, and it is a lot of fiddling around (there is only one stack you see).

I just put that caveat in the documentation in case someone tried it.

WillFa USA Wed 28 Jul 2010 05:55 AM #7

Twisol said:

Or:

t = {assert(CallPlugin ("80cc18937a2aca27079567f0", "show_message", "red", "green", "message", 42))}

Ugly enough to warrant a wrapper, perhaps:

do
  local orig = CallPlugin
  function CallPlugin(...)
    return {assert(orig(...))}
  end
end

t = CallPlugin("80cc18937a2aca27079567f0", "show_message", "red", "green", "message", 42))

assert is always true tho, because we're getting C-style error reporting, not a nil + error code Lua-style.

Nick Gammon Australia Forum Administrator Wed 28 Jul 2010 05:56 AM #8

Twisol said:

Or:

t = {assert(CallPlugin ("80cc18937a2aca27079567f0", "show_message", "red", "green", "message", 42))}

You can't assert, it returns 0 on success, not nil on failure, for backwards compatibility. You would need a wrapper that asserts on (return_code == 0).

Worstje Netherlands Wed 28 Jul 2010 06:00 AM #9

How difficult would it be to support Lua -> Non-Lua calls? If I get it right, we are only talking primitive types, which should be supported through the WSH APIs without much more trouble. Likewise, returning the value can be similarly solved by having a function return an array (list) rather than an int, and if that is the case unpack said array to end up with the Lua result.

Non-Lua -> Lua might be a bit harder, but even then I believe it possible without necessarily breaking compatibility.

Create a CallPluginEx(pid, func, params) function for non-Lua languages, the params parameter being typed as an array. Same primitive types stuff applies, of course. The result can be an array as well, an empty array meaning a nil-like value was returned.

Now, I even imagine it quite possible to have support for nested non-cyclic tables in Lua-Lua, and with a non-Lua plugin being involved that being limited to nested non-cyclic arrays. But that's nice stuff for the future if the basics are worked out right.

Twisol USA Wed 28 Jul 2010 06:02 AM #10

Ah, right. Well then:

do
  local orig = CallPlugin
  function CallPlugin(...)
    local results = {assert(orig(...))}
    if table.remove(results, 1) ~= 0 then
      return error(results[1]) -- return is idiomatic
    end
    return results
  end
end

t = CallPlugin("80cc18937a2aca27079567f0", "show_message", "red", "green", "message", 42))

Also, you can do this to "import" a function to use a more natural interface:

function ImportPlugin(id, name)
  return function(...)
    local results = {assert(CallPlugin(...))}
    if table.remove(results, 1) ~= 0 then
      return error(results[1]) -- return is idiomatic
    end
    return unpack(results)
  end
end

Foo = ImportPlugin("80cc18937a2aca27079567f0", "Foo")

Foo("Hello, world!")

Now -that- I like.

WillFa USA Wed 28 Jul 2010 06:04 AM #11

Nick Gammon said:

Anyone that does this:


CallPlugin (GetPluginID (), "foofunc", {1, 2, 3, 4})

... deserves all the pain he gets, honestly.

And why stop there?


CallPlugin (GetPluginID (), "CallPlugin", GetPluginID (), "foofunc", {1, 2, 3, 4})

Obfuscated Lua!


function wtf(...)
   return lpeg.Ct(lpeg.C(lpeg.P(1-lpeg.P(","))^1) * (lpeg.P(",") * lpeg.C(lpeg.P(1-lpeg.P(","))^1))^0 + lpeg.P(-1)):match(table.concat({CallPlugin(...)},","))
end

(You wanted LPEG...)

Nick Gammon Australia Forum Administrator Wed 28 Jul 2010 06:09 AM #12

Twisol said:

Ah, right. Well then:


...
 local results = {assert(orig(...))}
...

You gotta lose the assert. The first return value is never nil, nor false.

Nick Gammon Australia Forum Administrator Wed 28 Jul 2010 06:11 AM #13

WillFa said:



   return lpeg.Ct(lpeg.C(lpeg.P(1-lpeg.P(","))^1) * (lpeg.P(",") * lpeg.C(lpeg.P(1-lpeg.P(","))^1))^0 + lpeg.P(-1)):match(table.concat({CallPlugin(...)},","))

My eyes! ....

Nick Gammon Australia Forum Administrator Wed 28 Jul 2010 06:14 AM #14

Worstje said:

How difficult would it be to support Lua -> Non-Lua calls?

Once defined, the prototype for a OLE function is not supposed to change. So I can't change it from returning a long to returning a table. Or accepting a table instead of a string.

Maybe CallPluginEx could be defined in a different way. Maybe we encourage everyone to use Lua.

As I said, you don't *have* to use the new method. The old one survived for quite a few years. And you have Twisol's language-neutral PPI as a fallback.

Twisol USA Wed 28 Jul 2010 06:23 AM #15

Factor out the table...

local process_results(code, arg1, ...)
  if code ~= 0 then
    return error(arg1, 2)
  end
  return arg1, ...
end

function ImportPlugin(id, name)
  return function(...)
    return process_results(CallPlugin(...))
  end
end

Foo = ImportPlugin("80cc18937a2aca27079567f0", "Foo")

Foo("Hello, world!")

No messing directly with tables. I like this code!

Twisol USA Wed 28 Jul 2010 06:24 AM #16

Nick Gammon said:

Twisol said:

Ah, right. Well then:


...
 local results = {assert(orig(...))}
...

You gotta lose the assert. The first return value is never nil, nor false.

Yeah, I accidentally forgot to remove it. ^_^;

Nick Gammon Australia Forum Administrator Wed 28 Jul 2010 06:26 AM #17

Yes, that is getting there. Even this:


function ImportPlugin(id, name)
  _G [name] =  function(...)
    return process_results(CallPlugin(...))
  end
end

ImportPlugin("80cc18937a2aca27079567f0", "Foo")

Nick Gammon Australia Forum Administrator Wed 28 Jul 2010 06:28 AM #18

Er:


function ImportPlugin(id, name)
  _G [name] =  function(...)
    return process_results(CallPlugin(id, name,  ...))
  end
end

ImportPlugin("80cc18937a2aca27079567f0", "Foo")

Worstje Netherlands Wed 28 Jul 2010 06:29 AM #19

Nick Gammon said:

Worstje said:

How difficult would it be to support Lua -> Non-Lua calls?

I don't see how the prototype would change. There would be two ways to deal with it...

# Python
def MyFunc(a, b, c):
  pass

... could only be called with 3 parameters. If more or less are used, it would be valid to throw an error.

Alternatively...

# Python
def MyFunc(params)
  a, b, c, d = params    # unpack the values
  pass

... should work just fine. On WSH level, one would probably use an array-like object as being the 'params' in question. This way, there would be no need to worry about any signatures changing.

Nick Gammon Australia Forum Administrator Wed 28 Jul 2010 06:42 AM #20

It's not just the number of arguments, it's the type.

I could get away with it in Lua, because it isn't constrained by the rules for OLE objects.

Worstje Netherlands Wed 28 Jul 2010 06:48 AM #21

I'm familiar with the types used in OLE / COM stuff, although it's been a few years. I am pretty sure there are ways to return non-ints, such as an array which, once more, could contain the multiple return values. E.g.

# Python, skipping the obvious type checks for clarity
def MyFunc(params):
  a, b, c, d = params
  return [a + b, c + d]

Perhaps it takes an out-declared parameter to return non-ints (I doubt that tho). I don't recall the specifics, but I am about 99% certain the concept I am talking about is possible.

Twisol USA Wed 28 Jul 2010 06:57 AM #22

Nick Gammon said:

Er:


function ImportPlugin(id, name)
  _G [name] =  function(...)
    return process_results(CallPlugin(id, name,  ...))
  end
end

ImportPlugin("80cc18937a2aca27079567f0", "Foo")

I'm not a fan of forcing the user to use a global in general, so I prefer actually returning the value and letting them set it as they wish. It's not really that much more typing, and you can create a wrapper around ImportPlugin to do it if you must.

WillFa USA Wed 28 Jul 2010 07:03 AM #23

I actually agree as well... plus, if my calling plugin has a Foo function, I don't want to trample it because I don't have the option of using "otherFoo = ImportPlugin()"

Slightly different question... I don't remember how you changed dependency loading... Does PluginA parse, PluginA calls OnPluginInstall(), PluginB parses... or do they all parse first before executing OnPluginInstall (so if a plugin does a broadcast in its install, dependent plugins have a chance to catch it)

Twisol USA Wed 28 Jul 2010 07:14 AM #24

He didn't change it, it still parses and runs OnPluginInstall at the same time. OnPluginListChanged was added as a mitigating factor.

Nick Gammon Australia Forum Administrator Wed 28 Jul 2010 07:17 AM #25

Worstje said:

I'm familiar with the types used in OLE / COM stuff, although it's been a few years. I am pretty sure there are ways to return non-ints, such as an array which, once more, could contain the multiple return values.

You can do all that, you just are not supposed to change the interfaces to a COM object, once it has been released. So I can't make something that used to take a string argument today, take a table argument tomorrow.

Twisol USA Wed 28 Jul 2010 07:20 AM #26

You could create a new function CallPluginEx. :)

Worstje Netherlands Wed 28 Jul 2010 07:36 AM #27

I did say CallPluginEx from my very first post where I suggested it, didn't I?

I need to find the bold button so people actually read my blurbs, I bet. First Twisol and now you, I feel like an old record. =)

Nick Gammon Australia Forum Administrator Wed 28 Jul 2010 07:59 AM #28

That *was* Twisol. However you did indeed mention it in your first post.

Twisol USA Wed 28 Jul 2010 08:04 AM #29

Hmm, I don't remember suggesting CallPluginEx before now. My claim to fame in this thread is the ImportPlugin wrapper.

Bast Wed 28 Jul 2010 02:22 PM #30

Hey all,

What is the max length for a string. I ask because if I serialize a table, what is the max size I can send through callplugin?

Bast

Worstje Netherlands Wed 28 Jul 2010 02:46 PM #31

I don't think believe there is any practical limit. Lua supports strings as long as memory allows, so at most it would imo depend on the WSH kit and the other languages. And I _think_ that the string type used in COM has a 32 bit length field, so that would imply 2^32-1 length is the maximum.

Or in other words - what the hell are you doing? In my experience when you ask what the maximum is for something, you are going about things the wrong way, and need to explain what brought you to ask the question so people can help you find a better solution.

Edit: Ooh, my guess was right on the spot according to http://www.codeguru.com/forum/archive/index.php/t-242083.html

Nick Gammon Australia Forum Administrator Wed 28 Jul 2010 10:58 PM #32

Bast said:

What is the max length for a string. I ask because if I serialize a table, what is the max size I can send through callplugin?

CallPlugin, in its new incarnation of Lua-to-Lua, won't use the WSH at all, so the real issue is "what is the maximum Lua string length?".

I suspect, but can't find a definite answer right now, that 2^31 bits is indeed likely to be it. I hope you weren't asking with the intention of using the maximum (which is around 2 Gb) because you are likely to fill up your memory.

Especially as to push the string from one script space to another, it would be making a copy of it. And then there would be the intermediate pieces that go up into the serialization process.

However if you are asking "can the string be more than 100 bytes?" the answer is "yes it can".

Nick Gammon Australia Forum Administrator Wed 28 Jul 2010 11:06 PM #33

A quick browse of the source seems to indicate that it could possibly go up to size_t bytes, which in our case is probably an unsigned 32-bit integer.

So basically, unless you are doing something, ah, crazy, it should be able to hold whatever you want to put into it. For really large amounts of data, it might be sensible to consider holding data in a SQLite3 database instead.

Bast Wed 28 Jul 2010 11:20 PM #34

No, I do not intend to send a 2GB of text to another plugin. I was just wondering because I came across a similar limitation in mousemenu.

In my miniwindows, I have a popup menu for the mouse that is autogenerated from plugin variables and commands and it hit a limit of some sort (which I did not measure) so I had to break the menu up into 2 menus.

Bast

Nick Gammon Australia Forum Administrator Wed 28 Jul 2010 11:56 PM #35

I'm not sure what that limit was, but I increased the number of menu items you can display from 30 to 100 in version 4.55.

The limit of 30 seemed sensible with flat menus, but with the nested menus a higher limit might be desirable.