ATCP plugin for Achaea

Posted by Nick Gammon on Tue 09 Mar 2010 01:45 AM — 49 posts, 179,604 views.

Australia Forum Administrator #0
The plugin below uses the new telnet subnegotiation built into recent version of MUSHclient to intercept ATCP messages from IRE MUDs such as Achaea.

Simply install it, and it will do a BroadcastPlugin when various messages are detected. These can be picked up by other plugins (such as experience bars, health bars, mappers).

Template:saveplugin=ATCP_NJG
To save and install the ATCP_NJG plugin do this:
  1. Copy the code below (in the code box) to the Clipboard
  2. Open a text editor (such as Notepad) and paste the plugin code into it
  3. Save to disk on your PC, preferably in your plugins directory, as ATCP_NJG.xml
    • The "plugins" directory is usually under the "worlds" directory inside where you installed MUSHclient.
  4. Go to the MUSHclient File menu -> Plugins
  5. Click "Add"
  6. Choose the file ATCP_NJG.xml (which you just saved in step 3) as a plugin
  7. Click "Close"
  8. Save your world file, so that the plugin loads next time you open it.



<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE muclient>

<muclient>
<plugin
   name="ATCP_NJG"
   author="Nick Gammon"
   id="85f72d0e263d75df7bde6f00"
   language="Lua"
   purpose="Nick Gammon's ATCP plugin"
   date_written="2010-03-09 10:04:32"
   requires="4.50"
   version="1.0"
   >
<description trim="y">
<![CDATA[
Install into IRE games to catch ATCP messages.

Other plugins can detect ATCP messages like this:

function OnPluginBroadcast (msg, id, name, text)
  if id == "85f72d0e263d75df7bde6f00" then
  
    if msg == 1 then
      do_ATCP_vitals (text)      -- eg. "H:496/496 M:412/412 E:1380/1380 W:960/960 NL:89/100 "
                                 --      health    mana      endurance   willpower experience
    elseif msg == 2 then
      do_ATCP_room_brief (text)  -- eg. "Continuing on the Parade of Zarathustra"
    elseif msg == 3 then
      do_ATCP_room_exit (text)   -- eg. "n,s"
    elseif msg == 4 then
      do_ATCP_room_number (text) -- eg. "401"
    end -- if   
 
  end -- if ATCP message
end

]]>
</description>

</plugin>


<!--  Script  -->


<script>
<![CDATA[

local CLIENT_ID = "MUSHclient " .. Version ()
local IAC, SB, SE, DO = 0xFF, 0xFA, 0xF0, 0xFD
local ATCP = 200


-- agree to use ATCP
function OnPluginTelnetRequest (type, data)
  
  if type == ATCP and data == "WILL" then
    return true
  end -- if
  
  if type == ATCP and data == "SENT_DO" then
    Note ("Enabling ATCP.")
    SendPkt (string.char (IAC, SB, ATCP) .. 
           "hello " .. CLIENT_ID .. "\n" ..
           "auth 1\n" ..
           "char_name 1\n" ..
           "char_vitals 1\n" ..
           "room_brief 1\n" ..
           "room_exits 1" ..
           string.char (IAC, SE)) 
     return true
  end -- if ATCP login needed (just sent DO)
  
  return false
  
end -- function OnPluginTelnetRequest

-- we got authorization request, eg. 
-- Auth.Request CH\n<identstr>
-- Auth.Request ON

function got_auth_request (s)
   
  -- calculate challenge response
  local function atcp_auth (seed)
    local a = 17
    local i = 0
    local n
    
    for letter in string.gmatch (seed, ".") do
      n = string.byte (letter) - 96
      if bit.band (i, 1) == 0 then  -- even/odd?
        a = a + n * bit.bor (i, 13)
      else
        a = a - n * bit.bor (i, 11)
      end -- if
      i = i + 1
    end -- for
    return a
  end

  if s:upper () == "ON" then
    Note ("ATCP authorization accepted.")
    return
  end -- if
    
  local identstr = string.match (s, "^CH\n(.+)$")
  
  if identstr then
    SendPkt (string.char (IAC, SB, ATCP) .. 
             "auth " .. 
             tostring (atcp_auth (identstr)) .. 
             " " .. CLIENT_ID .. 
             string.char (IAC, SE)) 
    return
  end -- if
  
end -- got_auth_request


function got_vitals (s)      -- eg. "H:496/496 M:412/412 E:1380/1380 W:960/960 NL:89/100"
  BroadcastPlugin (1, s)
end -- got_vitals

function got_room_brief (s)  -- eg. "Continuing on the Parade of Zarathustra"
  BroadcastPlugin (2, s)
end -- got_room_brief

function got_room_exits (s)  -- eg. "n,s"
  BroadcastPlugin (3, s)
end -- got_room_exits

function got_room_number (s)  -- eg. "441"
  BroadcastPlugin (4, s)
end -- got_room_number

handlers = {
  ['Auth.Request']  = got_auth_request,  -- handled here

  ['Room.Num']      = got_room_number,
  ['Room.Brief']    = got_room_brief,
  ['Room.Exits']    = got_room_exits,
  ['Char.Vitals']   = got_vitals,
  
  } -- end handlers
  
function OnPluginTelnetSubnegotiation (type, option)

  if type ~= ATCP then
    return
  end -- not Achaea subnegotiation

  local command, args = string.match (option, "^([%a.]+)%s+(.*)$")
  
  if not command then
    return
  end -- don't seem to have a command
  
  local f = handlers [command]
  
  if f then
    f (args)  -- call handler
  else
    BroadcastPlugin (0, option) -- other, just send whole message
  end -- handler
          
end -- function OnPluginTelnetSubnegotiation

]]>
</script>


</muclient>

USA #1
Several perfectly functional ATCP plugins have already been written, one of which I currently maintain, an early version of which you even (kindly) host here. Was there a pressing need for you to reinvent the wheel? If you just wanted to showcase the telnet subnegotiation facilities in MUSHclient, fine, but I would have (and still will) updated mine.

I'll also point out that there are plenty of plugins already written that depend on my version. Two ATCP plugins can't coexist without starving one of them, and thus their dependents.
Amended on Tue 09 Mar 2010 02:01 AM by Twisol
Australia Forum Administrator #2
Twisol said:

Several perfectly functional ATCP plugins have already been written, one of which I currently maintain, an early version of which you even (kindly) host here. Was there a pressing need for you to reinvent the wheel? If you just wanted to showcase the telnet subnegotiation facilities in MUSHclient, fine, but I would have (and still will) updated mine.

I'll also point out that there are plenty of plugins already written that depend on my version. Two ATCP plugins can't coexist without starving one of them, and thus their dependents.



"Several perfectly functional ATCP plugins have already been written" ... so, re-writing an ATCP plugin isn't a new idea?

I was in fact trying to demonstrate how to use the telnet subnegotiation callbacks rather than mucking around at the packet level detecting things like IAC IAC which is really a low-level client job.

I took some care to examine the existing ATCP plugin and make identical BroadcastPlugin messages. I have modified slightly the other two plugins I just posted to work with your existing ATCP plugin, or the one I posted here (basically just checking for either plugin ID in the OnPluginBroadcast function).

Testing shows they work with either one, so I haven't really introduced a new, incompatible, system.

I have to say I couldn't work out by looking at the code what your plugin does with the Room.Num message, so I made up a new code for the (message number 4).

There is no particular reason for people to switch to my version here if they are happy with what is working for them, and in any case, as I said, I tried to make it compatible.


USA #3
I was thinking more in chronological order. My version is the latest in a series of improvements on the same base. Then you also have people like - who was it, WillFa? - who wrote their own personally. But mine is, AFAIK, the most known and used revision.

As for how mine works, it doesn't care what messages are recieved or count on the existence of specific messages (except Auth.Request). It just packages up a message and sends it to all plugins that said they wanted it, through my PPI library. It's purely a protocol-level plugin, offloading all data to interested parties.

I wrote the internal telnet parser because, at the time, the subnet functionality wasn't entirely fleshed out, and I was waiting for 4.50 and OnPlugin_IAC_GA to rewrite it.

Also, my version (not the one you have hosted) doesn't even use BroadcastPlugin anymore. The current version is on a page on my blog, which I linked to you a while back.

Lastly, presenting multiple implementations of ATCP will only confuse users, and it puts the burden on the dependant-plugin author to support both versions. It's better for the community as a whole to have a clear concensus in situations like these.
USA #4
Just to reiterate, because this is very important, the version you have of my plugin is vastly out of date, and is incredibly different compared to the one I have now. This is probably my fault for not letting you know about the changes, but I did link you to my most recent version a bit ago too. So please take another look! Thanks.
Australia Forum Administrator #5
Where? I couldn't find it at:

http://github.com/Twisol

I'm getting lost now, you may have sent me links before but my old brain is having trouble keeping track of the distinction between the "Twisol" stuff hosted here (which you specifically mentioned above), your GitHub files, your forum postings here, and other things hosted in a different place again.

Short of going back and re-reading I-don't-know-how-many forum messages, I can't really work out where the latest stuff is.
USA #6
I linked it to you at the beginning of our email correspondence. ;) I'd get you a direct link, but I'm on the iPhone and it's hard right now. You can go to Jonathan.com and click the plugins link at the top, though. I'll edit a link in here when I'm home. (10 minutes-ish)
USA #7
http://jonathan.com/?page_id=29

All of the ATCP-based plugins I've made are hosted there, barring my MapWindow plugin which I'm happy to deprecate in favor of your new mapper.

EDIT: By the way, I notice that on GitHub you made a commit for compatability with Trevize's version. Trevize has officially turned over his ATCP-based plugin development to me, his ATCP versions are no longer available, and mine are directly based off of his. In other words, it's not worth the trouble ;)
Amended on Tue 09 Mar 2010 04:18 AM by Twisol
Australia Forum Administrator #8
Twisol said:

It’s pretty interesting to see how a program I’ve used for a long time actually functions (even though he uses the Whitesmiths indentation style, which I cannot abide.


I love that style.
USA #9
Nick Gammon said:

Twisol said:

It’s pretty interesting to see how a program I’ve used for a long time actually functions (even though he uses the Whitesmiths indentation style, which I cannot abide.


I love that style.


Not only does it feel really weird to me, VS2005 defaults to Allman and I don't know how to change it (nor, honestly, do I want to). But that was really for comedic effect anyhow. :D
Australia Forum Administrator #10
Twisol said:

By the way, I notice that on GitHub you made a commit for compatability with Trevize's version. ... In other words, it's not worth the trouble ;)


Where did I say that?

[EDIT] Oh sorry, I read "commit" as "commitment".
Amended on Tue 09 Mar 2010 04:25 AM by Nick Gammon
Australia Forum Administrator #11
Twisol said:

Not only does it feel really weird to me, VS2005 defaults to Allman and I don't know how to change it (nor, honestly, do I want to). But that was really for comedic effect anyhow. :D


Don't worry, probably more duels at dawn over indentation than anything else.

;)
Australia Forum Administrator #12
Twisol said:

Two ATCP plugins can't coexist without starving one of them, and thus their dependents.


Why?

I notice once I installed your plugin mine was indeed starved of the telnet sequences but why did that have to be?

Plugins were intended to be independent of other plugins, so that installing one should not crimp the style of another. I see in the code for main.lua of your atcp.plugin download you have this:


OnPluginPacketReceived = function(packet)
  return parser.parse(packet)
end


This effectively seems to be stripping out the telnet sequences. But why do that? I changed it to:


OnPluginPacketReceived = function(packet)
  parser.parse(packet)
  return packet
end


... and my plugins then worked as they were intended to. And yours would too would they not? Why make a plugin that reduces the amount of data available to other plugins?

USA #13
Nick Gammon said:

... and my plugins then worked as they were intended to. And yours would too would they not? Why make a plugin that reduces the amount of data available to other plugins?



Because it reveals - or used to, I suppose, with the new subnegotiation support - some ATCP messages. Specifically Char.Vitals, which for some reason was actually displayed on-screen when the ATCP plugin was disabled or crashed (mostly during development and testing) after ATCP was requested from the server.

Also because, as I mentioned in another thread, ATCP requires a hello message listing the modules you want to activate. You can only send one 'hello' message, and this one 'hello' needs to know all of the modules that are to be used. There's just no good way around this in a distributed fashion. My current solution is to enable all modules and allow dependent plugins to simply ask for specific messages as they come in. This has the excellent bonus that dependent plugins that are added after you've already connected are highly likely to still work perfectly fine.

EDIT: I've updated my ATCP plugin now to replace my parser with the subnegotiation callbacks. I'm just checking that everything works properly and I'll upload the new version. But my point about 'hello' still stands firm.
Amended on Tue 09 Mar 2010 05:38 AM by Twisol
USA #14
The conversion was pretty quick and simple, I just replaced the parser callbacks I wrote with the associated OnPlugin callbacks. The new version is up at [1], but there's no major difference in how it actually operates.

[1] http://jonathan.com/?attachment_id=98
USA #15
At some point, I'd like to be able to come to an agreement or compromise over the ATCP plugin, because I believe that there should be only one generally available. I'll be uploading all of my plugins to my own repository on GitHub shortly, including my ATCP plugin, so it should be easier to compare and discuss.
Australia Forum Administrator #16
I'm all for agreement. ;)

It seems to me after testing your newer version that they don't clash as much as you are no longer stripping out the telnet negotiation from the raw packets. We still have the issue of multiple "hello" messages however.
USA #17
Nick Gammon said:

I'm all for agreement. ;)

It seems to me after testing your newer version that they don't clash as much as you are no longer stripping out the telnet negotiation from the raw packets. We still have the issue of multiple "hello" messages however.


That's a protocol issue, though. The protocol itself states that the hello must be the first ATCP message sent by the client (besides enabling ATCP in the first place), so unless we have a way to unify outgoing 'hello' messages in a usable manner, it's not something you can easily do. Obviously it would be best if every plugin that used ATCP could stand on its own, but the client itself would need to provide an interface specifically for ATCP to make that feasible.


At that point, you have only two real options:

1. Standardize a single ATCP plugin and make it easily available. This is my preferred choice.

2. Incorporate full ATCP support into the client itself. I dislike this; it's more of a gut feeling than anything, but once we have ATCP, why not ZMP or some other arcane protocol? ATCP is only used in IRE games, remember. And even if we do decide to do this, the API needs to be consistent, clean, and useful. To quote someone in a book I've been reading, we have to have the empathy gene. How will it be used? Does it make sense? This is something I'd rather not decide in a hurry, this should be fleshed out.


So far we're doing fine with ATCP as a plugin, so 2 is more work for little gain in the end. And we can always, always refine a plugin, much more easily than a core client interface. I'm also a big fan of doing as much as possible with client extensions; I think a lot about MUSHclient would be better off as a plugin, like the Convert Clipboard Forum Codes utility and the activity toolbar.
Australia Forum Administrator #18
Twisol said:

The protocol itself states that the hello must be the first ATCP message sent by the client (besides enabling ATCP in the first place), so unless we have a way to unify outgoing 'hello' messages in a usable manner, it's not something you can easily do.


If you can find something that documents the protocol that much you are doing better than I am.

In any case, since this is the sticking point, I have written to the Achaea admins suggesting that this requirement could be relaxed. What real purpose does it serve? Basically you are just asking the server to enable certain messages. Why does this all have to be done at once?
USA #19
Good question. I'll admit I tend to stay away from pestering them, mainly because I don't have the kind of clout required to make it not sound like whining. If there was a way to simply enable/disable messages at any given time, and simply start out as "ATCP enabled, all modules disabled", that would certainly help a lot.

On the other hand, I do know that modifications to Rapture (the MUD engine they use) have to go through more red tape than modifications to the code of a single game. I can't recall who said that specifically - maybe Kunin, the Tears of Polaris producer? - but I suspect that ATCP is a core part of Rapture.
USA #20
Nick Gammon said:

Twisol said:

The protocol itself states that the hello must be the first ATCP message sent by the client (besides enabling ATCP in the first place), so unless we have a way to unify outgoing 'hello' messages in a usable manner, it's not something you can easily do.


If you can find something that documents the protocol that much you are doing better than I am.


http://www.ironrealms.com/nexus/atcp.html
ATCP Documentation said:

To facilitate the flexible messaging protocol, we define a single message that is "outside" the module system and is provided to allow the client to tell the server about what modules it supports. This should be the very first message the client sends. (Note: currently there is no way to tell the -client- what features the server does, but this could easily be added by simply filtering the list and providing the last step in the negotiation.)

It goes on to explain exactly how the 'hello' message is structured. I've known about this page for a long time, though, no worries. It took a lot of Googling to actually dig it up in the first place.

It's a bit out of date when it comes to the most recent messages, like Client.Map and Room.Num, though. It would be nice if they could release a more up-to-date revision, and in fact that's one of the things I did ask them about. Never really heard back about it though.
Amended on Wed 10 Mar 2010 06:05 AM by Twisol
USA #21
I've uploaded all of my ATCP-based plugins to GitHub, including my current ATCP plugin. They all follow the "structured plugin" paradigm, which allows them to keep close the libraries and resources they need. The main advantage is that the PPI library that my ATCP plugin uses to communicate with the other plugins doesn't have to be downloaded separately, since it's included with each plugin.

http://github.com/Twisol/MUSHclient-plugins
Australia Forum Administrator #22
This is what troubles me ...

My experience bar plugin is 151 lines, and the ATCP plugin I wrote is 204 lines. The ATCP plugin can be shared between the experience bar, the health bar and the mapper.


You want me to replace the ATCP plugin with a downloaded zip file which contains 8 files and 4 folders. The files are:

  • plugin.xml - 20 lines
  • aliases.lua - 9 lines (seems to be for debugging)
  • plugger.xml - 47 lines
  • ppi.lua - 390 lines
  • reflex.lua - 321 lines
  • main.lua - 240 lines
  • parser.lua - 174 lines


Total for getting ATCP functionality: 1,201 lines.

Then I have to work out how it all works. And I gather with the PPI stuff I then have to "register" the desire to receive messages (my method just uses BroadcastPlugin).

Your system may be great for a more complex environment, but I want to be able to just get up a health bar or experience bar without all this fiddling around. And have the files available right here (or on GitHub which I control).


Twisol said:

Also because, as I mentioned in another thread, ATCP requires a hello message listing the modules you want to activate. You can only send one 'hello' message, and this one 'hello' needs to know all of the modules that are to be used.


Actually after discussing with their programmer, you can send more than one, and it respects the *last* one - I think you gave the impression before that only the first one counted. I explained about the multiple plugin problem and they are happy to remove that restriction, and allow the "hello" messages to be cumulative.

The whole thing about "if you install my plugin, then your plugin won't work" goes against the grain of the design of plugins (http://www.gammon.com.au/plugins/) where they are supposed to be "self-contained" things that don't require a lot of cross-plugin functionality to work.

I admit I have moved the ATCP functionality to a second plugin myself, but it is designed in such a way that it should not interfere with other plugins - apart from the IRE problem with only one "hello" message which I only heard about this week, and am on the way to solving.
USA #23
Nick Gammon said:
Total for getting ATCP functionality: 1,201 lines.

Then I have to work out how it all works. And I gather with the PPI stuff I then have to "register" the desire to receive messages (my method just uses BroadcastPlugin).

Your system may be great for a more complex environment, but I want to be able to just get up a health bar or experience bar without all this fiddling around. And have the files available right here (or on GitHub which I control).

I have to say that I disagree with your reasoning here. I don't know anything about Twisol's implementation so my comment is not a statement regarding his work. That said, your reasoning is essentially saying that special purpose libraries are always the correct solution. Yes, if you just want a health bar and some stuff like that, maybe something small is appropriate. But eventually people will want more (or they already do want more) and it rapidly becomes unmanageable to have a bunch of special-purpose components all trying to do the same thing.

Now, again to be clear, I'm not arguing for or against Twisol's implementation (I don't know anything about it). I'm arguing against using lines of code and exact specific applicability alone as metrics, because it seems very clear to me that when designing a general-purpose library, you need to be, well, more general.
USA #24
Nick Gammon said:

This is what troubles me ...

My experience bar plugin is 151 lines, and the ATCP plugin I wrote is 204 lines. The ATCP plugin can be shared between the experience bar, the health bar and the mapper.

You want me to replace the ATCP plugin with a downloaded zip file which contains 8 files and 4 folders. The files are:

[...]

Total for getting ATCP functionality: 1,201 lines.

Line count is an absolutely awful way to go about it. I value structure above and beyond mere brevity. You also list parser.lua, which I removed in this last release.

(EDIT: Also note that the majority of my plugin is dedicated to the PPI-based interface, which is mere fluff on the core functionality. The PPI-based interface makes it a lot easier (and perhaps even more fun) to work with ATCP.)

Nick Gammon said:
Then I have to work out how it all works. And I gather with the PPI stuff I then have to "register" the desire to receive messages (my method just uses BroadcastPlugin).

I admit I haven't gotten my documentation up to scratch yet, but I always point to roomname.plugin as an example of how to use the interface. You do have to register for specific messages, but it's no different from, say, registering event handlers with Javascript. (That might be a bad example, I don't know, but it's not an uncommon technique)

Nick Gammon said:
Your system may be great for a more complex environment, but I want to be able to just get up a health bar or experience bar without all this fiddling around. And have the files available right here (or on GitHub which I control).

What fiddling around? I think setting up your own ATCP interface is more work than adding another plugin and using an API that's already been shown to work in general. I think the biggest issue is understanding the "structured plugin" format. You're right: it's a zip file. It's structured internally to allow for easily breaking up your own plugin into manageable components, and bundling resources and third-party libraries to make it a single unit. (Just like compiling sqlite3, zlib, png, etc. into .libs and linking them statically into the MUSHclient executable.) It takes a little getting used to on the development side, but it maintains the one-download aspect of normal XML plugins while making it way easier on the developer to include and manage other files required for the plugin to work.


Nick Gammon said:
Actually after discussing with their programmer, you can send more than one, and it respects the *last* one - I think you gave the impression before that only the first one counted.

I admit the details are fuzzy - the server's treatment of 'hello' isn't explained in the spec - but in the end it's the same: you have to unify the 'hello' messages in the end, or all the previous ones won't count.

Nick Gammon said:
I explained about the multiple plugin problem and they are happy to remove that restriction, and allow the "hello" messages to be cumulative.

Whether the unification occurs client-side or server-side doesn't matter, so I'm very happy about this. Assuming I can send multiple 'hello's, and each one can enable new modules ones while maintaining the status of the previous ones, that's fantastic. A way to -disable- messages would be great too, but...

Nick Gammon said:
The whole thing about "if you install my plugin, then your plugin won't work" goes against the grain of the design of plugins (http://www.gammon.com.au/plugins/) where they are supposed to be "self-contained" things that don't require a lot of cross-plugin functionality to work.

This plugin has existed in various forms for a long time. It was a very iterative development, and before the changes to telnet subnegotiations in MUSHclient, some of them leaked through (as I mentioned with Char.Vitals). It was essential to strip the ATCP data out to guarantee that the data wouldn't show up on-screen. Nowadays you've fixed that particular issue.

On a note of personal ideology, I think you're being a little narrow-minded about plugins. What you envisioned plugins to be doesn't matter, it's how plugins are actually used that matters. (Look at the many uses of WD-40 for a good analogy.) Obviously, plugins that are standalone tend to be better, because it's a single download to get it working. But that's no reason to say that they can't or shouldn't work closely with other plugins, because you can do some really cool things that way.

Nick Gammon said:
I admit I have moved the ATCP functionality to a second plugin myself, but it is designed in such a way that it should not interfere with other plugins - apart from the IRE problem with only one "hello" message which I only heard about this week, and am on the way to solving.

Again, my ATCP plugin was developed iteratively over a long, long period, and I'm only the most recent of its maintainers. I'm very glad that I can now remove (and have removed) the blocking behavior of the ATCP data. Once the 'hello' message has been solved, the ATCP plugin very well may not be needed anymore... though I'd still be glad to maintain an auxiliary library for ATCP to make it easier, that you can stick in a structured plugin. ;)
Amended on Wed 10 Mar 2010 09:57 PM by Twisol
Australia Forum Administrator #25
(in response to David Haley):

What I am saying is, it seems too much. Basically I am trying to match certain sequences that comes down from the MUD, like a trigger. Now you don't install 1200 lines of code to match a prompt do you?

Really, the code needed to pull out room numbers is as simple as this:


-- agree to use ATCP
function OnPluginTelnetRequest (type, data)
  if type == 200 and data == "WILL" then
    return true
  end -- if
end -- function OnPluginTelnetRequest

function OnPluginTelnetSubnegotiation (type, option)

  if type ~= 200 then
    return
  end -- not Achaea subnegotiation

  local command, args = string.match (option, "^([%a.]+)%s+(.*)$")
  
  if command == "Room.Num" then
    handle_room_number (args)
  end -- if
          
end -- function OnPluginTelnetSubnegotiation


I don't really see why that couldn't even be in every plugin, it's not as if the protocol is going to change every 5 minutes.

This reminds me of the time I wanted to do something simple, like break up a URL into its component parts (http, the domain, the directories and the file, that sort of thing).

Now I read somewhere that the Boost library did that stuff, and in the interests of not "reinventing the wheel" I looked for it, and downloaded it (it was PHP code I think).

Then when I tried to run it I was told I needed to get ZEND as well, so quite a few megabytes later I had installed ZEND. Then when I tried to run it I was told I needed some "dependency" libraries, so quite a few more megabytes later I had downloaded and installed those. Then each of those libraries had more dependencies, and there was more downloading.

So after about an hour, and about 100 megabytes later, it still wasn't working and I called a halt to it. In my brain I thought, "this 'little library' is bigger than my entire application!".

It would have been quicker, and easier, and I would have understood what I was doing, if I just wrote my own regexp that broke up the URL. So that is what I did. I reinvented the wheel.

It's a similar thing with stuff like Python - to distribute Python with MUSHclient would more than double the size of the distribution. So the "little script engine" is bigger than the application it is installed in. Lua, however, is not.
Amended on Wed 10 Mar 2010 10:14 PM by Nick Gammon
USA #26
(I only spotted the "in response to David Haley" after I wrote most of this, so I figured I might as well post it anyways. Grain of salt.)

Well, yes. It's a juggling contest. That whole way-overdone dependency tree is one of the things my structured plugin design is intended to solve. You can include the libraries you want with the plugin that needs it (unless it's provided by MUSHclient natively, obviously). ATCP uses three libraries:

1. Plugger, a stub library which makes it easier to write a structured plugin by "fixing" the Lua paths to be relative to your plugin's folder instead of the MUSHclient plugins/ folder. It also automatically executes scripts/main.lua, creating a divide between plugin metadata (the .xml) and implementation (whatever other .lua files you have).

2. Reflex, which provides a natural Lua-oriented interface to triggers, aliases, and timers. It's an alternative to the XML format, and being something of a purist in that respect, I opted to use Reflex to implement my debug alias. You mentioned perhaps including it with MUSHclient at some point, which would mean ATCP (and everything else, really) wouldn't have to include their own copy.

3. PPI (my version), to facilitate easier and more natural communication between plugins. Specifically, it lets other plugins register callbacks to specific messages, which makes it more modular and neater.

Dependency hell? No, because they're included explicitly in the plugins that need them. What would be four downloads to get a working ATCP plugin - three if you want to drop Reflex - is taken down to one in an easy, simply-unzip package.


(EDIT: To the code snippet you posted:) In theory, I agree with you that it should be that simple. But one important point is that many Achaeans use the Vadi system, which is a proxy you connect to on your computer. At the time of this writing, it sends its own 'hello' message and doesn't pass your own through, and it misbehaves if you do try to send one. So I have an extra check in my OnPluginTelnetRequest to -not- send a 'hello' message if we're connected to 127.0.0.1. And that's something the general developer could easily forget, although then it would be their fault and not mine.
Amended on Wed 10 Mar 2010 10:25 PM by Twisol
USA #27
I guess the issue is that Twisol's ATCP handler is meant to operate in a framework, and that framework requires some other stuff. Is the framework is always necessary? Well, probably not. Does that mean it's not useful? I don't think you could say that either.

Basically if the argument is merely "look, mine is smaller" I don't think that suffices. It would have to be an awful lot smaller to justify discarding a framework for writing more plugins.

Interesting questions here might be to what extent the ATCP stuff actually needs the rest of the stuff (like PPI) and whether or not the ATCP core could be isolated, and the framework integration provided separately.

FWIW I disagree that that code should be repeated in every plugin, even if yes the protocol won't be changing every five minutes. Repeating code like that is simply a bad idea. What if the protocol does eventually change? All of a sudden you need to update every single plugin that uses that functionality. This seems to me to be a fairly obviously inferior situation to having a slight overhead the first time you install a plugin.

I agree that often it feels like too much when you just want a simple thing; Xerces felt like this for me for XML parsing. "I just want an XML tree..." But comparing this to Xerces or Zend or PHP seems like a little bit of a stretch to me.
USA #28
David Haley said:
Interesting questions here might be to what extent the ATCP stuff actually needs the rest of the stuff (like PPI) and whether or not the ATCP core could be isolated, and the framework integration provided separately.


Mmm... I could remove Reflex if I wanted, since it only powers one minor debugging alias. I could also remove Plugger if I wanted, but I like to use it because it helps enforce the plugin's structured-ness, and it's so few lines anyways.

PPI is how the data actually gets sent around, and it's integral to the registered-callbacks facility. That's not so much a framework as simply a way to easily get at the data. I've gone the BroadcastPlugin route before - that's how it originally was - and it was ugly, and it was very hard for the client to tell the ATCP plugin to enable a module or to get a specific message or whatnot in a clean manner.

EDIT: I'm not sure what context you're using "framework" in, now that I think about it. Are you referring to my "structured plugin" format, or the ATCP functionality itself?
Amended on Thu 11 Mar 2010 05:40 AM by Twisol
#29
I agree with Nick on two things here: these two plugins are different takes on the same thing and can co-exist peacefully, and the simple solution is often the preferred solution.

The PPI code is a means of passing data back and forth between plugins, which I find largely unnecessary. I steer away from plugins as much as possible in my own coding, preferring to have everything in the same shared space. Things like this feel too much like COM to me, and those of us who've dealt with COM programming know the pain it can cause.
USA #30
Larkin said:
I agree with Nick on two things here: these two plugins are different takes on the same thing and can co-exist peacefully, and the simple solution is often the preferred solution.

In theory, yes. In practice, there are obstacles, but we're slowly breaking those down.

Larkin said:
The PPI code is a means of passing data back and forth between plugins, which I find largely unnecessary. I steer away from plugins as much as possible in my own coding, preferring to have everything in the same shared space. Things like this feel too much like COM to me, and those of us who've dealt with COM programming know the pain it can cause.

I'll suggest that you find PPI unnecessary precisely because you avoid plugins anyways. It becomes necessary for any rich, meaningful interactions between plugins. I don't know what you do, personally, but I write things that I intend to be used by other people, and plugins are indispensable for that. (We've tried having people import sets of XML and add code to their scripting file, and so many people had trouble with it!)

A shared scripting space would probably be better for this in some ways, but having toyed with scripting in WoW, which does this, it's also a bit unsatisfying. I'm working on a sort of half-and-half architecture in my own hobbyist web client, which I think shows promise, but obviously it can't be applied to MUSHclient this late in the game. I just do the best I can with the tools I'm given.
Amended on Thu 11 Mar 2010 05:34 PM by Twisol
#31
For smaller things that are easily modularized, I'm sure it's very useful. For a full-fledged combat system, it's just impractical to pass so many bits of data back and forth between so many modules.

I provide fairly simple instructions to using my scripts, and nearly everyone has zero trouble getting it all installed. I've seen things that are much more plugin-based and had a hard time understanding them myself, not to mention the typical end user who wants to do a few add-ons of his own and doesn't even know where to begin.

To each his own, of course.

And, these new Telnet negotiation ATCP plugins are very helpful! Thanks for the updates, guys. :)
Australia Forum Administrator #32
I got a message today from the programmer at IRE saying that on Imperium Imperian at least, multiple "hello" messages are now respected. The changes may take a few days to flow through to the other IRE games.

Some good has come out of our discussions about ATCP. A limitation that previously had to be worked around, perhaps somewhat laboriously, has now been removed.

Indeed the outcome of most discussions here, even if they may not satisfy everyone, usually result in improvements which benefit all users.
Amended on Thu 11 Mar 2010 07:36 PM by Nick Gammon
USA #33
Nick Gammon said:
I got a message today from the programmer at IRE saying that on Imperium at least, multiple "hello" messages are now respected. The changes may take a few days to flow through to the other IRE games.

Excellent. I'll look into experimenting later this week, figure out how best to react to the new treatment of 'hello'.

Nick Gammon said:
Imperium

Imperian. ;)
USA #34
Quote:
PPI is how the data actually gets sent around, and it's integral to the registered-callbacks facility. That's not so much a framework as simply a way to easily get at the data.

Well, no, it kind of is a framework: to get at the ATCP stuff you need the PPI framework, instead of the normal BroadcastPlugin-whatever-it-is. It's not clear to me that this is a bad thing, however it means that you're necessarily getting more than just an ATCP handler.

Quote:
EDIT: I'm not sure what context you're using "framework" in, now that I think about it. Are you referring to my "structured plugin" format, or the ATCP functionality itself?

The structured plugin is another framework thing. ATCP on its own is not really a framework. A framework is a collection of stuff to make some functionality available. You basically have a framework on top of MUSHclient's framework (its standard broadcast function).



Anyhow my point here is really not to argue for or against any particular implementation. My point was that using lines of code alone is a somewhat short-sighted metric. Sometimes more is less, but sometimes more really is more.
Australia Forum Administrator #35
Twisol said:

Imperian. ;)


Oops.
Australia Forum Administrator #36
David Haley said:

Anyhow my point here is really not to argue for or against any particular implementation. My point was that using lines of code alone is a somewhat short-sighted metric. Sometimes more is less, but sometimes more really is more.


Well it is time to roll out Gammon's Law:

Quote:

The solution cannot be simpler than the problem.


What that means in essence is that you cannot have a simple solution to a complex problem. Just as an example, if you are going to serve a three-course meal, you need to cook (or otherwise obtain) three different lots of food.

However a corollary may be:

Quote:

The solution should not be more complex than the problem.


In essence, I agree a complex solution (and therefore to a certain extent, lines of code) is required to solve complex problems. However I worry when I see what seems to me to be complexity addressing a simple problem.
USA #37
Nick Gammon said:
Well it is time to roll out Gammon's Law:
Quote:
The solution cannot be simpler than the problem.


In other words: Simpler than the problem (is) not a solution.
Contrapositive: A solution (is) at least as complex as the problem. (EDIT: More correctly, "not simpler than the problem", but it comes to the same.)

Nick Gammon said:
Quote:
The solution should not be more complex than the problem.


Your corollary is less a corollary and more of an additional refinement. It doesn't follow directly from the original law ;)
Amended on Thu 11 Mar 2010 08:01 PM by Twisol
USA #38
Well, even if we grant your law unconditionally, we come to the next problem which is that there is apparently disagreement on what the problem is in the first place.

I think you are being somewhat reductionist when you say that the only problem is to display a health bar. The problem is bigger than that: it's serving up data sent by the server to plugins to do as they will. Some of those plugins might have only simple needs.

Heck, if we look at things only ever so slightly differently, isn't the entire plugin architecture far too complex for this? You could just as easily stick something into the core that intercepts ATCP and puts up a small window with the health bar. It would probably be simpler than the whole plugin architecture, too. So why do we even bother with plugins?

Of course, that solution is far less flexible, and fairly obviously inferior to what we have now.

My point is solely that although your needs today, for the purposes of your example code, are simple, you haven't really argued the case that your needs will always be so simple. Your solution of simply duplicating the code where needed is a little surprising to me, because it's a pretty clear violation of other software design principles.
USA #39
David Haley said:
Your solution of simply duplicating the code where needed is a little surprising to me, because it's a pretty clear violation of other software design principles.


Granted, my solution also duplicates code; specifically, the libraries the plugin depends on that aren't included with MUSHclient.
USA #40
Well, IMHO that's wrong too. It would be silly for every plugin that uses PPI to have to reimplement PPI. Of course, PPI is the kind of concept that should make it into the core somehow (that discussion kind of fell off the wagon, as it were). The structured plugin stuff is a similar concept that should be in the core. (I use 'concept' to differentiate it from a particular implementation that may or may not be suitable.)
Australia Forum Administrator #41
Twisol said:

Your corollary is less a corollary and more of an additional refinement. It doesn't follow directly from the original law ;)


True.
USA #42
I've been popping into Achaea a few times every day, and I just checked now, and they've got the new ATCP messages added! Assuming the 'hello' message changes were also applied, I'll probably be tinkering with my ATCP layer soon. I'd like to create an ATCP library (as opposed to plugin, now) that can be included in structured plugins, if things work out.
USA #43
I just wrote up a small ATCP library. It seems to work quite well, and in a similar fashion to my original ATCP plugin.

Minor problem though, Nick: If I recall, you said that only the plugin that sent the DO gets called with "SENT_DO", right? I think that should be changed in favor of calling all plugins with SENT_DO. WILL would short-circuit at the first plugin to enable it, of course, but any number of other plugins might need to know if it's been enabled, and do their own setup relevant to the protocol. This is particularly an issue with the ATCP library, since it needs to be able to send its "hello" message to guarantee that those modules are enabled.

Here's what I have so far. It's probably not finished - I'd like to fix up the module infrastructure a bit - but it seems to work in testing.

local codes = {
  IAC_DO_ATCP = "\255\253\200", -- enables ATCP communication
  IAC_SB_ATCP = "\255\250\200", -- begins an ATCP packet
  IAC_SE      = "\255\240",     -- ends an ATCP packet
  ATCP        = 200,            -- ATCP protocol number
}

local hello_msg = "hello MUSHclient " .. Version() .. "\n" ..
                  "auth 1\n" ..
                  "char_name 1\n" ..
                  "char_vitals 1\n" ..
                  "room_brief 1\n" ..
                  "room_exits 1\n" ..
                  "map_display 1\n" ..
                  "composer 1\n" ..
                  "keepalive 1\n" ..
                  "topvote 1\n" ..
                  "ping 1\n"

local callbacks = setmetatable({}, {
  __index = function(tbl, key)
    local group = {}
    rawset(tbl, key, group)
    return group
  end,
})


local Register = function(msg, callback)
  if type(msg) ~= "string" then
    error("Message name must be a string")
  elseif type(callback) ~= "function" then
    error("Callback must be a function")
  end
  table.insert(callbacks[msg], callback)
end

local Unregister = function(msg, callback)
  if type(msg) ~= "string" then
    error("Message name must be a string")
  elseif type(callback) ~= "function" then
    error("Callback must be a function")
  end
  
  for k,v in ipairs(callbacks[msg]) do
    if v == callback then
      table.remove(callbacks[msg], k)
      break
    end
  end
end

local Startup = function(type, data)
  if type == codes.ATCP then
    if data == "WILL" then
      return true
    elseif data == "SENT_DO" then
      SendPkt(codes.IAC_DO_ATCP .. codes.IAC_SB_ATCP .. hello_msg .. codes.IAC_SE)
      return true
    end
  end
  
  return false
end

local Handle = function(type, data)
  if type ~= codes.ATCP then
    return
  end

  local message, content
  local separator = data:find("%s")
  if separator then
    message, content = data:sub(1, separator-1), data:sub(separator+1)
  else
    message, content = data, ""
  end
  
  for _,callback in ipairs(callbacks[message]) do
    local ok, err = pcall(callback, message, content)
    if not ok then
      TraceOut("Error while executing ATCP callback " ..
               tostring(callback) .. " for message " .. message ..
              ":\n  " .. (err or ""))
    end
  end
  
  -- send to catch-all handlers as well
  for _,callback in ipairs(callbacks["*"]) do
    local ok, err = pcall(callback, message, content)
    if not ok then
      TraceOut("Error while executing ATCP callback " ..
               tostring(callback) .. " for message " .. message ..
              ":\n  " .. (err or ""))
    end
  end
end

return {
  Register = Register,
  Unregister = Unregister,
  Startup = Startup,
  Handle = Handle,
}



Usage:
ATCP = require("atcp")

foo = function(message, content)
  -- ...
end,

bar = function(message, content)
  -- ...
end,

catchall = function(message, content)
  -- ...
end

OnPluginInstall = function()
  ATCP.Register("Char.Vitals", foo)
  ATCP.Register("Room.Name", bar)
  ATCP.Register("*", catchall) -- receives all messages
end

-- Set up ATCP handlers
OnPluginTelnetRequest = ATCP.Startup
OnPluginTelnetSubnegotiation = ATCP.Handle


-- Alternatively, if you want to handle these events yourself, too:

OnPluginTelnetRequest = function(type, data)
  ATCP.Startup(type, data)
  -- ...
end

OnPluginTelnetSubnegotiation = function(type, data)
  ATCP.Handle(type, data)
  -- ...
end
Amended on Sat 13 Mar 2010 05:45 AM by Twisol
Australia Forum Administrator #44
Twisol said:

Nick: If I recall, you said that only the plugin that sent the DO gets called with "SENT_DO", right? I think that should be changed in favor of calling all plugins with SENT_DO.


You are the one that said we couldn't have multiple logins, and so the SENT_DO is only sent to the first plugin. How are other ones going to know if someone else logged in before them?

Maybe you are right that it should be changed, but this shows what happens if you fiddle around too much trying to solve a problem caused by one particular server design.
USA #45
Nick Gammon said:

Twisol said:

Nick: If I recall, you said that only the plugin that sent the DO gets called with "SENT_DO", right? I think that should be changed in favor of calling all plugins with SENT_DO.


You are the one that said we couldn't have multiple logins, and so the SENT_DO is only sent to the first plugin. How are other ones going to know if someone else logged in before them?

You are the one who got them to relax the exact limitation that made me say that! XD

Nick Gammon said:
Maybe you are right that it should be changed, but this shows what happens if you fiddle around too much trying to solve a problem caused by one particular server design.

Personally, I think the subneg handling itself is more at fault than any given server implementation. They have every right to do what they want with their protocol, especially if it already conforms well to the telnet subneg specification. It's a bit pointless to discuss alternatives though, when it's too late to do anything about.
Australia Forum Administrator #46
Well I agree with you on that one. The telnet protocols are pretty good examples of people not thinking things through. For example, the whole issue of what happens when you get:


 IAC SB x blah blah IAC y 


where "y" is neither IAC nor SE, is undefined.

So that's example #1.

Then you have sub-negotiation loops, which I think I saw myself, where the server says:

IAC WILL 200

You reply:

IAC DO 200

The server then sees "IAC DO 200" and replies "IAC WILL 200". The client then says "IAC DO 200". The server replies "IAC WILL 200" and so on until the network bandwidth is all swallowed up.

It's an example of a protocol that is ill-thought through. The best they could come up with afterwards is, "ah don't get into negotiation loops". Well, der.
USA #47
Nick Gammon said:
It's an example of a protocol that is ill-thought through. The best they could come up with afterwards is, "ah don't get into negotiation loops". Well, der.


I remember Elanthis (a MudBytes user) saying that the "proper" way is to keep a table of values for all 255 possible subneg types, with three possible values: active, inactive, or unknown. And they all start at unknown. That's what you ended up implementing in MUSHclient, if I remember.

At any rate, comments on the library? The entire PPI layer was removed, as it's no longer needed, and replaced by the Register/Unregister functions.


As a side thought, it would be really great to talk to someone who maintains ATCP itself, just to get an idea of how, exactly, they expect clients to implement the protocol. What they expect the data to be used for. Etc. But most of the time in the history of ATCP, I believe we've been pretty much on our own.
Australia Forum Administrator #48
Twisol said:

I just wrote up a small ATCP library. It seems to work quite well, and in a similar fashion to my original ATCP plugin.


I haven't forgotten about this, but have had mapper improvements coming out of my ears today. Remind me in a few days if there is no further comment.