Future versions: Implementing Plugins

This conversation has been going on at length in another thread:

http://www.gammon.com.au/forum/bbshowpost.php?bbsubject_id=1276&page=999999

...but since much of the conversation there is not very relevant, I am starting this new thread.

The purpose of the thread is to discuss how "Plugins" should be implemented in future versions of MUSHclient.

(continuing previous conversation):

The reason I made the distinction between default files and other "include" files, is because default files and the "core"/"mother" world files would be meshed (with the "mother" files over-riding the Default's), where as an "include"/"child" files would be completely independant of it's "mother".

Mother + Default = Default Settings/Values over-ridden
Mother + Child = Settings/Values Co-exist in different namespaces.

The advantage to keeping the "mother" and "child" worlds (namespaces) independent, is that you naturally avoid collision of names and variable or setting values.

The disadvantage, is that cross-communication between a child [sub]world and it's mother could be difficult, and may require extensive [re]programming of the client.

Suppose, for example, you keep them seperate:

You may have a script file you use for all of your worlds, that takes output created by the script for the world, and logs it when sending:

Sub LogandSend (String)

If each include file runs in it's own namespace, then a second include script would not be able to call that subroutine in the first include script.

On the other hand, if a third include script has it's own built in "LogandSend" subroutine (that could be slightly different), then you may run into a problem if the include files are NOT run in their own names space.

Similar problems arise with identical variables, or triggers, aliases, timers with identical names.

(We started calling Variables, Aliases, Triggers, Timers "VATT" for short in the other thread).

It seems the solution may be to let each run in their own space, with special formatting/arguments made to functions to allow them to cross-communicate across the multiple "namespaces".

Yes, I agree. I thought that the default of separate namespaces might be useful in most cases, but probably should be overridable where you specifically want to communicate between them. For instance, the MXP room descriptions would be outside a namespace, as they are generated by the main client and not a plugin. I was thinking of something like this:


'
' Find my namespace ...
'
world.note world.get_my_namespace ' eg. P1:

'
' Access another namespace ...
'
world.note world.getvariable ("P2:target")

'
' Access a global variable - the leading colon means
' "the global namespace"
'
word.note world.getvariable (":mxp_roomdesc")

(or maybe put mxp variables into their own namespace)

'
' Access an MXP variable (if we did that)
'

word.note world.getvariable ("mxp:roomdesc")


This leads to another interesting point. Should plugins have names? Should those names be unique? If not, conceivably more than one person will make a "wilderness walker" plugin. If that happens, then how does a script find the namespace for a plugin? Maybe based on the file name? At least, the full pathname would need to be unique, but then, how could you predict what the full pathname will be on someone else's PC?

This is the problem that the GID (Global Unique Identifier) idea tries to solve. Perhaps that could be used here.

Each plugin could have a GID and that would uniquely identify it. I could add a script function to generate them (may as well use Microsoft's method). eg.


world.note world.generate_guid ' 12340005-4980-1920-6788-123456789012


Ugly though they may be, at least you could rely upon them to give uniqueness. Then you might see if another plugin was installed, like this:


world.note world.get_namespace_for_guid ("12340005-4980-1920-6788-123456789012") ' eg. "P14:"


Interesting (and annoying) point. :)

Hmm...

Offhand, your idea seems good, though ugly, as you said. I can't think of another method offhand that would be fool proof.

Can you use variables as part of a call to a subroutine, like that example above? I don't think VBS is set up like that. Would it work if I replaced the ":" with a "." ?

The only other thing I can think of, is that each plugin identify itself (in the XML file - I guess), but that would not be foolproof. I assume you would present an error to the user when they initialized the parent world (if there was a name collision). The user would then be left to handle it on their own, perhaps editing the Plugin XML file, or notifying it's author.

Hmm...

Oh, did you understand that part about Default files?

Default files get loaded into the namespace of the main world, Plugins do not.

I think this is a handy distinction to make, and allow through the continued use of Default files, rather than dropping them and going with Plugins only (which wouldn't have access to some of the settings available for Defaults).

...or did you intend that a default file would just be a plugin, with everything set for the main world:

Plugins would not be allowed to set that option... but personally, I would want a default file that could set any option.

Magnum,

The more I think about it I am inclined to go down your path of virtual machines, at least to an extent. I was previously thinking that plugins would share a "global" set of VATT (variables, aliases, triggers, timers) but would avoid name collisions by using a namespace prefix, so that "target" would become "P1:target" for plugin 1.

However this gets pretty tortuous, and then I start asking myself, do you want plugins to save their variables to the world file anyway? Do you want plugins to access/change world settings (eg. world.note colour)?

I am thinking now of moving the VATT lists so that each plugin would have its own lists, thus eliminating namespace problems without specifically having to have a name for them. This could have good side-effects, at possibly the expense of some limitations. How about this for rules for plugins ...

1. Plugins are installed in such a way that they are clearly identified as plugins, as opposed to include files. Include files would be for the user of MUSHclient to organise default colours etc. Plugins are for installing scripts written by other people. As such, you want to be protected a bit more from what plugins might do.
2. Each plugin has its own set of VATTs (variables, aliases, triggers, timers). Thus, getvariable, setvariable automatically work on the local list.
3. Plugins cannot access VATTs from other plugins, or the global set.
4. Changes made to VATTs by plugins are not saved. ie. plugins are a read-only resource. eg. they can add/change/delete triggers, variables etc. but this will be a temporary thing
5. Plugins are restricted from changing some global settings (eg. player name).
6. Plugins are restricted from reading some global settings (eg. player password)
7. Some (all?) things that plugins are allowed to change (eg. 'echo input' flag) are saved before the plugin is called and restored afterwards, so that they cannot permanently affect the environment (thus solving the problem of changing note colours etc.)
8. Plugins must have a unique ID (eg. a GUID) - although I am not sure that is necessary any more.
9. Because plugins would have separate trigger/alias lists, the trigger matching routine would have to "walk" each plugin. I'm not sure what order it would do this, perhaps the order the plugins were loaded. Thus, a high-priority trigger (eg. sequence 10) would only be high-priority for that plugin, not necessarily for all plugins.

Not exactly. I was expecting the namespace to be something like "p1:", not a COM reference, and you would use it for GetVariable, however see my previous post which obsoletes that idea a bit. Also, your example tends to assume that you would get a world object, which I was explaining isn't really practical.

Conceivably you might support this sort of syntax:


world.Execute ("P1", "fix_all_items")


This would find the namespace (virtual machine?) P1, and execute the command "fix_all_items" in its script engine.

However, I'm not sure about doing that.

If you restricted plugins to not allow the <world> tag, that might fix that.

Hmm... You said this in the other thread:

I'm not clear about this. Would this be an object?

Ultimately, I think it would be ideal if a scipt could run in either the global space or a plugin space. This would be the most versatile.

In order to do that, I can't help but refer back to the previous thread where we started this conversation.

If you could build a new type of object (lets call it a "plugin" object), that does essentially what I proposed, that might be ideal.

I'm really sorry and a bit frustrated that I don't understand objects perfectly, and COMs.

I wouldn't be surprised if you could build a new plugin object type (essentially cloning most of your global object code), but the tricky party would be doing the three essential things I listed before: Taking output from the global mud and calling the plugin muds code, and taking output from the plugin and sending it to the global object.

I'll put that aside, because I don't want to run in circles. From your numbered list:

2. Excellent, but would each plugin also have it's own settings? If so, you could have independent speedwalk settings, note colours, echo status, etc... (Clone as many settings as you can).

3. That's what I expected as well, though the idea listed above, about having access with an object prefix identifyer might still be a good idea, especially if you can work it to call scripts in other objects too. (We could work without this for a while, and look at it as something to add later, when the basic plugins are running)

4. This is a bit of a tough one to swallow. For example, my EQ script does not load the variables from it's file every time the client is restarted (though it could). There are plenty of other instances though, were it would be more convenient to have the values remain across sessions. My previous recommendation was to store the values back into the Plugin's XML file, though it would become "de-virginized" after doing so...

5. Hmm, I hate pointing out so many conflicts, but... I was thinking it might be nice to create a "login" script, which would probably be a GUI that lets you pick one of your player names, and then the script would log in for you using the appropriate password. Such a script would make it hard for another script to grab the player name, if there was no means to store it globally. Am I correct in that you could currently set another world's setting by doing something like "otherworld.setoption"?

7. If possible, clone and store these independantly, then each plugin could have it's own independant settings!

You said you can't create the plugin in a world object of it's own. I pretty much need to know, before I can make suggestions, can you make a new type of "plugin" object (which is almost the same as a global world object)?

Can you have nested objects? (One inside the other)?

I think you can see where I am going with this... I've already posted more than I intended... I don't want to run in circles, but I can see my own footsteps ahead of me...

Magnum lays across the mattress, staring at the wall... blankly. His eyes close but there is no rest. Suddenly, a light bulb turns on. Oh no! Not another brilliant idea.

I had this thought... I got out of bed and did a search on the internet for "nested object", and found what is probably the perfect example to look at for myself:

http://www3.sympatico.ca/gavin.mckenzie/rebol/xml-object-info.html

Perfect, if I understand it.

The Global Object contains information such as TCP/IP address, size of scrollback buffer, and other items unique to a connection.

The Enviro Object contains information such as Note Colour, Speedwalk Delay, Echo Status. These settings must be declared at least once, but each "Plugin" can then modify the values and have it's own independant versions. This Object is nested into the Global Object.

The Local Object are basically VATT information. Settings that are empty when a NEW world is established. This Object is nested into the Enviro Object.

Each Enviro Object will only have one Local Object nested inside it.
A Global object may have multiple Enviro Objects nested inside it. One of the Enviro Objects is considered the "Core" Objects belonging to the Main XML file. This lets you use the same load/initialize routine as you do for plugins, just a special case of loading/saving from main XML file.

An Enviro Object is basically your plugin, independant from others.

You save in reverse. Plugin data is stored back in the Plugin's XML file.

Workable? Would be nice. Too bad it probably would practically require a complete re-write of the client, and would not likely be backwards compatable. On the bright side, it would be a good time to do that name change to MUclient you've been thinking about. :)

Hmm.. Thinking about it there are also potential problems if a script expects that note color has a certain value but is reset when you leave that script.. If you employed something like Magnum's nesting idea then my main script wouldn't care what the 'current' state of the note color or similar settings are any more than the plugins would. That way there couldn't be any direct interference in either direction. However I do agree that some sort of 'save state' needs to exists for plugins or at least some way to tranfer values to or set root variables so that you can have persistance from one session to another. Without this you can end up with some major problems trying to 'plugin' many existing scripts or even use them properly. A nested system could even have different definitions for custom colors and not interfere with the main world settings, though I am not sure how your output buffer keeps track of what colors are being used in this case. However unlike the basic ansi settings, there is no real reason to not have different custom colors from one plugin to the next.

As for item 9... I wonder if a binary tree of some sort could make 'walking' the scripts easier at least for things like aliases, since you could have pointer entries added for the tree each time an alias gets added that say starts with a-c, then when something is typed you just hunt the tree for pointers to that first letter and 'only' check those plugins that contain such an alias. Unless someone creates aliases that use every number and letter possible at least once this would probably speed things up quite a bit. I can't see any easy way to do that for trigger unfortunately. :P

I wasn't envisaging it. :)

Let me clarify, that when I first started adding COM to MUSHclient, I wasn't really an expert either (probably still aren't) and if I was, might have done it somewhat differently.

For example, a trigger should probably be a COM object, with properties. So, this sort of thing might work (it doesn't now) ...


dim aTrigger

set aTrigger = world.GetTrigger ("x")

world.note aTrigger.Name ' trigger's name
world.note aTrigger.Match ' match text
aTrigger.Enable ' enable myself


You are right, what you are proposing would be a big change, and frankly I want to not spend weeks tweaking whether nor not something is a property of a COM object or not, but make changes that might actually be useful for people playing a MUD (eg. a HP status bar).

Hmm... I'm not sure where to go from here. Right this moment, the whole endevour is feeling a little futile. It would probably just be easier to write a standalone program to install plugins using the current "workaround" method. Indeed, if you wanted to take a vacation from this whole idea, I would understand...

If you wanted to, you could slowly move the client in that direction, casually refining code with "Plugins" something to aim for another time.

I don't like the idea of writing to plugin files. It seems to me the whole idea of include files (including plugins) is to share them between worlds. Thus, if you shared a plugin of any sort between two worlds, and it stored data in the plugin file (when? when the world saved?) then the only copy that really persists is the last one written.

Also, you have the issue of the plugin write routine having to reconstruct the plugin source (put the scripts back etc.).

No, no, I don't think that will work.

Maybe, have a "plugin save" file, where a plugin can remember its state. That might need a unique name, because you will have multiple plugins and multiple worlds. The mind boggles a bit thinking about it.

Probably simpler to let plugins save variables to the main world file, but then you have the issue of "your world file has changed - do you want to save it?" when you can't remember making any changes.

I think I can make something work quite soon, ie. this week. Then some keen plugin writers can test it. :)

For instance, the recent post I did for someone who wanted to tell his friend where some other player was, would make a nice little plugin.

Well.. For one if the plugin layer is in a seperate COM, then COM provides a means to save the state of the object anyway. Though not sure how or if it would work in this case.

As to the idea of saving it in the main world... It bugs me already that I have to tell the dang thing 'yes save the bloody variables' every time as it is. Some sort of persistance in needed though since some data 'must' be persistant in certain types of plugins and relying on writing the data to a file is not something everyone knows how to do. Not to mention that variables are persistant regardless, while a file needs to be intentionally updated, leaving you open to loosing the current state in process. This is one reason why saving the state probably wouldn't work anyway.

Maybe if it was simply explicitly stated that plugins must provide their own persistance methods, but... OF course on way to go would be to use a simpler ID method. Two bytes probably are sufficient, since who is going to plugin 65535 different pieces of code from other people. lol The use the ODBC to store the variables from plugins. All windows systems should have ODBC and a few basic database types installed by default, they have to or programs like Quickview wouldn't be able to view Excel files. Doing that would eliminate the saving of them into the world file and still support persistance, since each access would save/retrieve from the database. Just an idea anyway...

Either a way needs to exist or there has to be a specific statement to the effect that it isn't directly supported, but that makes such plugins a bit harder to use well, it also begs the question of what happens if you made the plugin and 'intended' it to be able to communicate with your main script. That is the most serious issue with regard to the problem.

You make some good points in that post, that I hadn't considered. You're absolutely right.

I suppose if you ever managed to adopt the nested objects idea, you could just store the whole Global Object, and all it's nested Objects in the main XML file, but that's a road that won't get travelled any time soon.

Like I indicated before, my brain is fried on the subject. I'll leave it in your hands for the time being, till I get a feel for where you are going with the Plugin idea, and how you are implementing it.

Well, as an amateur coder, i can say that most of this conversation is way over my head.
But in the spirit of contribution i thought i would say how i think that you should make the plugins work, in the moron's programming language of english.

First off, I've decided that the 'plugins' should be little program sort of things, e.g. item databases, world mappers, that sort of thing. In my magical world of plugins, the plugin would have three basic 'sections', 1.something to start it,
2.the thing that it does and
3.what it sends back to the world.
The start would be like a 'trigger' or alias to make it go off and to collect any data that it needs to run. Next, the plugin runs, does its thing. Finally, if it needs to, it sends stuff either to the mud window (eg if you change the look), or back to the actual mud.

In this sort of thing the plugin is like a parasite that leeches onto mushclient. It has its own little dropdown menu or tab or whatever, and under this it has other little areas for any triggers it might use, any variables it uses, a help file explaining it, this sort of thing. *IF* the plugin wants to get other triggers that arent part of it, well it should be able to, just let it do it via the existing script functions. I reckon that'll solve the problem of having the same names for triggers.

So this is what i think a plugin should be. i realise this probably does help but i think it sort of explains my suggestion on how to code it so the little morons like me can understand them and make their very own. Having said this, if i have grasped the wrong end of the stick, please tell me if im talking complete nonsense, i'd like to know :P

Well Dan.. I have put a bit of thought into making a sort of floating task bar sort of thing that could support such plugins as you are talking about and link itself into MUSHClient through triggers and scripting, however imbedding such a thing into MUSHClient directly has to potential to slow the client down a lot and thus is not likely to happen unless someone can 'prove' that it can be done without causing such a drop in efficiency and since this isn't opensource it won't happen. ;)

As to what we are discussing... We are talking about the ability to include other peoples scripts into our own worlds in a way that lets both our scripts and theirs to work as intended without interfering with one another. In this sense a 'plugin' is a piece of a world file with scripts, triggers and other things that are needed to use someone elses code, but loaded in such a way that they won't do anything but what they are intended to do. The problem is that there are a lot of things they need to do and many of those things have the potential of messing with stuff your own scripts and world files have no way to know have been changed by a plugin. The issue is how to allow them and still effectively prevent such conflicts. I believe we have resolved most of the issues, but there may still be a few issues that need to be resolved, but we are waiting to see how Gammon figures out how to fit things together before jumping in again to point out all the flaws. ;)

One big issue though is what to do if the plugged-in script 'needs' to be able to store settings or data and expects the clients world file variables to store that information. It is the only major thing we haven't really resolved in our discussions.

DanPrice,

You have described pretty closely how I intend plugins to work, so you are on the track. :)

Just as an update, I have most of that working now. A plugin (written in XML) implements the following:

• Triggers
• Aliases
• Timers
• Variables
• Scripts

All in their own space (namespace? engine?) so they don't clash with other MUSHclient things of the same name. Also, scripts can be in their own language (eg. one plugin in VBscript, another in Jscript, a third in Perlscript).

Most of the existing script commands will refer to their own namespace (eg. addtrigger), again to prevent clashes.

I haven't yet tackled "saving state", and in any case I think some plugins won't need to. By saving state I mean remembering from one invocation (of MUSHclient) to the next what it was up to (list of spells etc.).

However what seems reasonable is to allow a script to save its variables into a "plugin save file", on a plugin-world basis. What I mean by that is, since plugins might be shared between worlds, you need to save the state per plugin, per world. eg. if you have 5 plugins and 4 worlds you potentially have 20 states that need saving.

It all seems to work fairly well, and I am now implementing a GUI interface (plugin list) so you can add/remove plugins in an orderly way.

After some fairly extensive testing I should be able to let some keen scripters at it in the middle of next week. The sooner the better, as a lot of the replies to queries on the forum would be well done as a small plugin.