Exactly how bad is it to call Simulate instead of ColourNote anyway?

Posted by Fiendish on Tue 03 May 2011 07:20 PM — 28 posts, 97,062 views.

Fiendish USA Global Moderator Tue 03 May 2011 07:20 PM #0

I find myself in a situation where I need to be able to modify parts of the same colorized output line in several different triggers (zMUD converts really do care about #sub functionality). It seems like the only sane way to do that is by using Simulate to display a colored string instead of a series of ColourTells that fake the colorized output after omitting the original line, because notes and tells aren't processed by triggers. And yet every time someone has suggested using Simulate or asked about using Simulate instead of Note/ColourNote/ColourTell, Nick says "Simulate kills kittens!"

So how bad is it anyway? Has anyone had world-collapsing experiences with it? I think it's the only reasonable option here.

Nick Gammon Australia Forum Administrator Wed 04 May 2011 12:08 AM #1

Well, do you like kittens or not? Besides, I never said that. ;)

My problem with Simulate is, apart from the fact that it wasn't designed for that, is that it will probably not work reliably.

Effectively it is injecting the simulated text into the input stream being considered by the input-handling state machine.

Now state machines are not things I like to muck around with.

Consider this sort of scenario ...

We have an MXP tag, eg. <room> so we now have a start tag but not yet the stop tag (</room>). The function OnPluginMXPopenTag is called, which calls Simulate to inject new text into the input stream. Text which might close the tag, or do something to the MXP state. After that we keep processing the original line. Now everything is out of sync.

And it might not happen every time. If the thing triggering the Simulate call happens to be at the end of a packet, then the Simulated data might just neatly appear as if it was the next thing from the MUD. But in the middle of a packet, it might not.

What are you trying to do exactly? Even if you omit a line from output, it is still processed for triggers. Do you want to change (say) X to Y and then trigger on Y?

Twisol USA Wed 04 May 2011 12:27 AM #2

It sounds like he has a line that multiple triggers want to modify. If two triggers re-echo the line though, you have two new lines instead of the original. It sounds like he's trying to get around that by Simulate()ing the modified line and hoping the other triggers match it too.

If this is about color instead of replacement text, however, you -can- stack color changes. You need to have one trigger that matches the line and enables a trigger group, then each of the triggers in that group can match a specific piece of that line and change its color. Then you need a trigger at the end to disable the trigger group again.

Fiendish USA Global Moderator Wed 04 May 2011 03:13 AM #3

Hmm...
Aardwolf doesn't use MXP. Would that make some of these concerns go away?

So here's the much longer version of my question...

I have a plugin that reads text from the mud in the form of

Quote:

{chan ch=gtell}(Group) Ol' Dirty Fiendish: 'hmm pizza'

It does omit-from-output, then uses the following code to strip off the initial {...} part and then re-display the result as a series of ColourTells to simulate as if the {...} part had never been there.


-- strip out the tag
tag_length = string.find(styles[1].text,"}")
styles[1].text = string.sub(styles[1].text, tag_length+1)
styles[1].length = styles[1].length-tag_length

-- display
for _, v in ipairs (styles) do
    ColourTell (RGBColourToName (v.textcolour),RGBColourToName (v.backcolour),v.text)
end -- for each style run
Note ("")  -- wrap up line

The result is

Quote:

(Group) Ol' Dirty Fiendish: 'hmm pizza'

So far so good. Except now I'm trying to convert zMUD users, and one of the first things they ask is "How do I do #sub in MUSHclient?" And my response usually involves a lot of "well..." and "that's actually rather complicated". So I figured, since I'm such a smart guy, I can write a magic alias that would turn "#sub {*} {*}" into a new trigger, in a way similar to the #trigger and #alias aliases that have been discussed before on the forums*, with the proper code to actually achieve a style of inline substitution whereby all instances of %1 in the mud output are replaced with %2. And I can do it, no problem...except...

Now I'm in a situation where I need (at least) two different scripts modifying the same lines from the MUD, and I need them both to preserve color codes, and that doesn't seem possible unless I change all of that ColourTell stuff to Simulate instead.

* - (if you don't remember the relevant forum threads, see https://code.google.com/p/aardwolfclientpackage/wiki/CommandLineAliasesAndTriggers for my writeup for users on those two aliases)

When the user types


#sub {hello} {pizza}
#sub {friend} {pie}

they will get two triggers that SHOULD cause the line

Quote:

{chan ch=gtell}(Group) Ol' Dirty Fiendish: 'hello friend'

to be replaced, in concert with the plugin above, with

Quote:

(Group) Ol' Dirty Fiendish: 'pizza pie'

That seems not possible without Simulate. Issues of where to put color breaks for complicated coloring are unimportant right now.

Also, keeping these triggers out of a plugin is kind of important to me for this project, so even if there weren't another plugin involved fancy plugin-specific tricks would probably be a last resort. :\

Twisol USA Wed 04 May 2011 03:54 AM #4

I applaud your effort. :) It can be hard to untangle some parts of MUSHclient's API.

I would suggest using a single trigger that matches any given line (just ^ works fine, you can use %0 to get the whole thing), and checking all of the #subs inside it. You would need to write the checking and replacement logic yourself, but you do have access to the PCRE library from Lua (rex.*). Afterwards you can Note() the line back out.

Fiendish USA Global Moderator Wed 04 May 2011 04:24 AM #5

Twisol said:

I would suggest using a single trigger that matches any given line (just ^ works fine, you can use %0 to get the whole thing), and checking all of the #subs inside it. You would need to write the checking and replacement logic yourself, but you do have access to the PCRE library from Lua (rex.*). Afterwards you can Note() the line back out.

So any time anything ever wants to replace output it has to offload the task to some function somewhere else which keeps track of the entire list of all possible replacements that any plugin or trigger ever wants to do? And if anything ever doesn't use this mechanism I get duplicated output? That's not going to work.

Twisol USA Wed 04 May 2011 06:12 AM #6

Fiendish said:
So any time anything ever wants to replace output it has to offload the task to some function somewhere else which keeps track of the entire list of all possible replacements that any plugin or trigger ever wants to do? And if anything ever doesn't use this mechanism I get duplicated output? That's not going to work.

Sorry, that's what happens when you have a singleton resource. You need a mechanism that can control access to the line and make sense of the substitutions. It's a lot better than always duplicating output.

And I think Simulate() will put the line on the very end of the input stream, so your line will actually be out of order. Quite possibly bisecting another line if not all of it made it into the current packet.

Fiendish USA Global Moderator Wed 04 May 2011 06:43 AM #7

Twisol said:
And I think Simulate() will put the line on the very end of the input stream, so your line will actually be out of order. Quite possibly bisecting another line if not all of it made it into the current packet.

At first my response is that I'd much rather see Simulate act the same way that Note does. If Note after omit can get the line in the right place, why can't Simulate? Except perhaps using this would be quite dangerous since it allows for infinite reprocessing loops.

Then I think about what the zMUD documentation says about the #substitute command. It says that "Substituted strings are not processed by further triggers". Here, "substituted strings" means just the replacement portions, not the entire line. Which means that, if I extrapolate the meaning correctly, in fact zMUD is displaying the original line and then sending the original line to the triggers and overwriting the output with substitutions as the triggers process. I wonder if MUSHclient could be made to do the same.

Twisol USA Wed 04 May 2011 06:48 AM #8

Fiendish said:
At first my response is that I'd much rather see Simulate act the same way that Note does, except. If Note after omit can get the line in the right place, why can't Simulate. But perhaps this can be quite dangerous.

Well, the difference between Simulate and Note is pretty big. Beyond where the data is put (Simulate -> incoming data buffer, Note -> output window lines), Simulate()d data also has ANSI, MXP, etc. parsed. AFAICT, the only thing that comes before it is decompression. Changing Simulate would also break backwards compatibility in a pretty big way.

Nick Gammon Australia Forum Administrator Wed 04 May 2011 08:26 AM #9

Fiendish said:

When the user types


#sub {hello} {pizza}
#sub {friend} {pie}

When this user types that, I am curious. Let's say that "hello" has colour codes in it. eg.


h<red>e<blue>l<green>l<white>o<pink>

What does the substitution do? Does it discard the original colours? In the case of friend/pie it must because the words are not the same length.

Fiendish said:

... and I need them both to preserve color codes, ...

That's why I am interested in your answer.

Fiendish USA Global Moderator Wed 04 May 2011 08:38 AM #10

Nick Gammon said:

When this user types that, I am curious. Let's say that "hello" has colour codes in it. eg.


h<red>e<blue>l<green>l<white>o<pink>

What does the substitution do? Does it discard the original colours? In the case of friend/pie it must because the words are not the same length.

I asked some people to test that for me before posting the original query. According to them, zMUD printed pink pizza. Clearly this is not perfect color preservation inside the replaced bits (in fact, I would have chosen red instead of pink, but that's not a big deal), but I mean it needs to preserve colors outside of the replaced bits and do at least a reasonable attempt at coloring the replaced bits. If I can get a system that allows for multiple-substitution, what happens to individual color codes within a substituted word is something I'll just make up and specify as "expected to behave strangely".

Nick Gammon Australia Forum Administrator Wed 04 May 2011 08:45 AM #11

Twisol said:

It can be hard to untangle some parts of MUSHclient's API.

By design. Just joking.

Twisol USA Wed 04 May 2011 08:46 AM #12

I'm of the opinion that the final style run should be extended to fit the replacement text if it's bigger, or style runs should be discarded if the replacement text was shorter. So this:

h<red>e<blue>l<green>l<white>o<pink>

Would be this:

p<red>i<blue>z<green>z<white>a<pink>

And this:

fr<red>ie<blue>nd ni<green>ck

Would be this:

pi<red>e<blue> ni<green>ck

And this (hello -> hi):

hel<red>lo<green> there

Would be this:

hi<green> there

Of course, if that's too funky, I'd prefer Fiendish's answer of "red". :P

Nick Gammon said:

Twisol said:

It can be hard to untangle some parts of MUSHclient's API.

By design. Just joking.

Nick Gammon Australia Forum Administrator Wed 04 May 2011 08:48 AM #13

Fiendish said:

Hmm...
Aardwolf doesn't use MXP. Would that make some of these concerns go away?

Colour codes?

Quote:

When the user types


#sub {hello} {pizza}
#sub {friend} {pie}

So you want the trigger to fire on something that it did *not* receive from the MUD? This is surely redefining what a trigger does?

Fiendish USA Global Moderator Wed 04 May 2011 08:48 AM #14

Twisol said:

I'm of the opinion that the final style run should be extended to fit the replacement text if it's bigger, or style runs should be discarded if the replacement text was shorter.

Wait, do we get to pick how it should work? Then I agree with Twisol, the final style run should get extended if necessary, or styles should get dropped if necessary.

Quote:
So you want the trigger to fire on something that it did *not* receive from the MUD? This is surely redefining what a trigger does?

Not exactly. I just want to be able to have lots of triggers that modify the same line based on what was sent by the MUD. You'll note that the documentation for #substitute specifies that the replacement bits (not from the mud) will not themselves get sent to triggers for processing. This is where Simulate() fails, unfortunately.

Twisol USA Wed 04 May 2011 08:52 AM #15

Fiendish said:
Wait, do we get to pick how it should work?

I do, I'm writing a client. ~_^

Fiendish USA Global Moderator Wed 04 May 2011 08:53 AM #16

Twisol said:

Fiendish said:
Wait, do we get to pick how it should work?

I do, I'm writing a client. ~_^

Will it be compatible with MUSHclient scripts? :P
I kid, Nick! I kid!

Fiendish USA Global Moderator Wed 04 May 2011 09:12 AM #17

So hold on...Back to what you said earlier:

Quote:
And I think Simulate() will put the line on the very end of the input stream, so your line will actually be out of order. Quite possibly bisecting another line if not all of it made it into the current packet.

So I've been playing around, and I've gone and replaced the series of ColourTell/ColourNote calls with


Simulate(stylesToANSI(styles))

where stylesToANSI does what its name implies. And it appears to be working great for me. Everything is in the right place, and I'm not splitting lines down the middle. Granted I have a fast connection and a fast computer. Are you saying that this will work differently for other people on the same MUD, or is it server dependent, or what? I'm not at all familiar with the packet level of MUD transmission.

Twisol USA Wed 04 May 2011 09:27 AM #18

Fiendish said:

So hold on...Back to what you said earlier:

So I've been playing around, and I've gone and replaced the series of ColourTell/ColourNote calls with


Simulate(stylesToANSI(styles))

Nick will be able to confirm or deny the inner workings of Simulate, but I'll explain the whole packet transmission thing.

You probably know that MUDs communicate using the Telnet protocol. Hower, Telnet sits on top of TCP, which is a "reliable, stream-based protocol". Stream-based means that you as a user do not deal with packets, you deal with a stream of data. Data you pass to a TCP socket is buffered up, and some/all of that data is sent as a packet. On the other end, incoming packets are buffered up, and you request chunks of data in whatever size you want. This means that if the server sends "Take me to your leader", a variety of things can cause the client to process "Take me to" and " your leader" in separate passes.

Compare this to UDP, which is an unreliable datagram protocol. It doesn't guarantee that data will be received in order, or even at all, and you deal directly with discrete packets of data. However, a datagram is never split up.

When I mentioned bisecting a line, I was talking about part of a line getting processed separately from the rest. If Simulate() appends data to the buffer of data currently being processed, it effectively wedges inbetween two chunks of incoming data. This kills kittens.

When I talked about a Simulate()d line being displayed out of order, I meant that it's very likely that one chunk of data will contain multiple lines. By appending the Simulate()d data to the end of the chunk, you're moving that line past the others.

It's possible that Simulate() has its own, separate buffer that's only processed when there's no more data available, but the vagaries of the 'net makes that an unreliable technique. (Better than the alternative, though - at least it works most of the time.)

Nick Gammon Australia Forum Administrator Wed 04 May 2011 10:29 AM #19

Twisol said:

When I talked about a Simulate()d line being displayed out of order, I meant that it's very likely that one chunk of data will contain multiple lines.

I agree with Twisol here.

As a thought experiment, imagine that each character from the MUD arrives separately, possibly separated by minutes of delay. Now in practice it isn't that bad, but when you propose design changes, make sure they are compatible with that model.

Fiendish USA Global Moderator Wed 04 May 2011 06:46 PM #20

So this all seems very unfortunate for me. I'm reading that it means I can't do what I want to do here.

Twisol USA Wed 04 May 2011 06:51 PM #21

Not built-in, and not the way you wanted to do it, no. There's still the virtualized trigger I suggested before; in fact I wrote and tested it already last night. I was going to finish my stylestring implementation before posting it, but why not:

stylestring = {}

function stylestring.new(str, styles)
  return setmetatable({
    str = str,
    styles = styles,
  }, {
    __index = stylestring,
  })
end

function stylestring.tell(self)
  local str, styles = self.str, self.styles
  local left = 1
  for _,t in pairs(styles) do
    local fg = RGBColourToName(t.textcolour)
    local bg = RGBColourToName(t.backcolour)
    ColourTell(fg, bg, str:sub(left, left + t.length - 1))
    left = left + t.length
  end
end
function stylestring.note(self)
  self:tell()
  AnsiNote() -- not sure if Note() preserves the current color, but I know this does
end

function stylestring.replace(self, match, replace)
  -- TODO: Stretch/shrink styles as needed.
  -- Until then, this is a glorified gsub.
  local str = self.str
  local match_len = match:len()
  
  local left = 1
  local mid = str:find(match)
  while mid do
    if left then
      str = str:sub(left, mid-1) .. replace .. str:sub(mid + match_len)
      left = mid + match_len
    end
    mid = str:find(match, left)
  end
  
  self.str = str
  return self
end

local subs = {}
function add_sub(match, replace)
  table.insert(subs, {
    match = match,
    replace = replace,
  });
end

function do_sub(str)
  for _,t in ipairs(subs) do
    str = str:replace(t.match, t.replace)
  end
  return str
end

function get_subs()
  return subs
end

<triggers>
  <trigger
   enabled="y"
   ignore_case="y"
   keep_evaluating="y"
   match="^.*$"
   omit_from_output="y"
   regexp="y"
   send_to="14"
   sequence="100"
  >
  <send>local str = stylestring.new("%0", TriggerStyleRuns)
do_sub(str):note()</send>
  </trigger>
</triggers>


<aliases>
  <alias
   match="^\s*#sub\s*{((?:[^}]|\})+)}\s*{((?:[^}]|\})+)}\s*$"
   enabled="y"
   omit_from_log="y"
   regexp="y"
   send_to="12"
   omit_from_output="y"
   ignore_case="y"
   sequence="100"
  >
  <send>add_sub(("%1"):gsub("\\}", "}"), ("%2"):gsub("\\}", "}"))</send>
  </alias>
</aliases>

Nick Gammon Australia Forum Administrator Thu 05 May 2011 04:14 AM #22

Fiendish said:

... one of the first things they ask is "How do I do #sub in MUSHclient?"

Well that is like the people that ask where is Lua's switch statement.

If I may suggest, doing a #sub isn't all that critical. I mean, when you are fighting mobs you don't think "oh I wish I could do a #sub"!

What you *might* think is "I want better stats" or "have I seen this mob before?" or "is that one of my friends?".

So I suggest the emphasis should be pulled back from "how can we #sub?" to "how can we solve the underlying problem?".

Now one thing that springs to mind is making a generic trigger (ie. it matches "*").

Then you feed that trigger into your own matching system that might change A to B, and then C to D and so on.

Twisol USA Thu 05 May 2011 04:22 AM #23

Nick Gammon said:
Now one thing that springs to mind is making a generic trigger (ie. it matches "*").

Then you feed that trigger into your own matching system that might change A to B, and then C to D and so on.

In other words, what I just implemented. ;)

Nick Gammon Australia Forum Administrator Thu 05 May 2011 04:33 AM #24

Quote:

match="^.*$"

Yeah, exactly like that. :)

Fiendish USA Global Moderator Sat 07 May 2011 05:22 PM #25

Twisol said:

I was going to finish my stylestring implementation before...

So stylestring is just an object that also contains the non-styled string representation of a style run?

Twisol USA Sat 07 May 2011 06:28 PM #26

Well, you could describe it that way. I think of it more as a normal string including style information. You need the whole string in one place to be able to do replacements properly, and I just kind of ignore the 'text' field in each style.

FenceWalker USA Mon 09 May 2011 05:01 AM #27

Fiendish said:

Then I think about what the zMUD documentation says about the #substitute command. It says that "Substituted strings are not processed by further triggers".

The documentation for CMUD says the same thing about #sub. However if you were to make a trigger in either CMUD or ZMUD (not sure about the free version I tested in 7.21), that took a string and substituted it with something else. Then made another trigger to fire on the substituted string, you'll see that the documentation for #sub is incorrect. Triggers WILL fire on #sub'ed lines.

Just wanted to correct that small bit of misleading information that is getting quoted from outdated help files.