RegExp VM

I am breaking this off from the thread "General: Triggers across lines", since it is getting a tad insane over there. lol

Having had a bit of a sleepless night, where I kept running to the computer having remembered something I forgot, I eventually came up with this:

VM operation/parts:

CodeArray - Holds the array of 'commands' that get executed.
RegExpArray - Holds the chunks of RegExp the preparser seperates out.
PC - Integer holding the 'program counter' that keeps track of the current instruction to be executed.

The PC always starts at 0 initially, but increases by 1 for each successful RegExp match. When such a match happens, the VM returns with a TRUE value and whatever wildcards where generated for the line. The next time the trigger is tested, it starts execution and the 'current' PC or if the PC is now greater than the number of instructions, it is reset to 0 and testing starts over from there.

Instructions and logic:


Examples of how the pre-parser would build the execution code for these:

Example 1 "^You see\:$(.*\n){1,10}^$">

test 1
rst
opt 2, 0
lp 10, -1, 3

Example 2 "^Fred says: .*">

test 1

Example 3 "^You see\:$(.*\n){1,}^$">

test 1
test 2
opt 2, 0
test 3

Example 4 "^You see\:$(.*\n){5,}^$">

test 1
rst
test 2
lp 5, -1, 0
opt 2, 0
test 3

Example 5 "^You see\:$(.*\n){5,10}^$">

test 1
rst
test 2
lp 5, -1, 0
rst
opt 2, 0
lp 10, -1, 3

Example 6 "^You see\:$(.*\n){,10}">

test 1
rst
opt 2, 0
lp 10, -1, 0

Example 7 "^You see\:$(Nothing\n)|(.*\n){1,}">

test 1
opt 2, 1
opt 3, 0
lp 0, -2, 0

Now.. If I could just find someone to design the needed pre-parser. lol But this does do both what you want Nick and follows the actual syntax, without gludging extra settings and a look-back buffer on to make it work, which is what I don't like. Unfortunately designing the pre-parser is much more complicated when you have to deal with situations like example 7. :( The rest are relatively straight forward.

Oh.. And I also considered using a set of 'opcodes' that included "test, inc, cmp, beg, bne, lp, ret <boolean>, rst", but realized that this wasn't really necessary, since all the logic really needed could be incorperated directly into the test, opt, inc and lp functions themselves.

I know you are trying to solve a problem here, but I don't think writing our own regexp machine will be a great idea. Apart from the "reinventing the wheel" aspect, I see from the PCRE changelog that the first documented item was in September 1997, and the most recent one in May 2003. That is almost 6 years of development to get a nice, stable, powerful routine.

The problem is, the only thing I think you really solve is not having to "count the lines". Other issues, like omitting from log, or output, would still apply, unless you are going to defer *all* trigger processing if a multi-line trigger happens to be active.

For the trivial case of (say):

You see Nick\nHe is hungry.

Sure, I can count the \n characters and automatically compute 2 as the number of lines, but as soon as it changes to:

You see (Nick\n)*He is hungry.

... I can't. I don't know how many times the \n will repeat.

Then if you want to write a regexp parser that takes streaming text "on the fly" then you have the problem it may never terminate. Here is an example:

(?s)^You see:(.*)

The (?s) says to consider the dot character to include newlines.

Now since this never terminates, it could absorb 1000 lines of output and still be going strong. With my approach you at least have to put a "limiting count". Say you chose 20, then it terminates after 20 lines, no matter what is in the output.

If anyone is looking for it, the previous thread was Triggers across lines.

Umm. I am not looking to replace the PCRE. PCRE simply isn't able to deal with text on the fly, which is why you are messing with a look-back in the first place. As far as normal regexp triggers are concerned there is 'no' change, it just makes the call with the current line and the only regexp segment the VM can retrieve, instead of calling it directly.

For triggers that match multiple lines, each *line* gets split out and treated as a seperate call to the existing PCRE system, the only difference is the breaking up of stuff that matches on multiple lines into a set of smaller regular expression that don't need to test all 10, 20, 50, 5000 or how ever many lines they match in order to succeed. Each pass through the triggers functions as though the trigger was only looking for 'that specific line' which the VM knows it needs to test. Yours can only do as many lines as you ask, since by definition, it is limited to the number of lines in the look-back buffer you can test against, the bigger the buffer, the more overhead and the slower the client gets. This isn't practical.

This doesn't replace the PRCE code you already use, it just does a bit of prework, so you can get around the fact that the mud text isn't a static file, while yours 'forces' it to be static, but cannot work unless the number of lines is >= to the number you tell it to expect. Mine adds a tiny bit of everhead to every regexp as they are tested, lets just say 10 clock cycles to make a guess. Assuming that testing each line in yours also took 10 cycles, yours automatically adds 10 * N, where N is the number of lines in the buffer it has to recheck every single time the trigger is tested.

The clear difference is in how the data is treated. From the perspective of users, mine works like any other trigger. Each time it matches something it make a call to the script engine or whatever. Yes, as you pointed out this does mean you have to deal with the data 'as it arrives' and store it in some cases, but that is the way people already do it. Their is no real reason why you couldn't, instead of testing the buffer every single time, just add to one that would get returned as the entire matched text. Then you get the same result as yours, but people can *choose* if they want to deal with the data when it arrives, store it themselves or use a split command to pull the stuff out.

In most cases I wouldn't even care what the entire matched text was. I am probably looking for specific information on each line and I want to deal with them when they happen, not after the fact, when I may not even have a clue which wilcard the 55th item is even in, given a complex case where there may be say 4 different types of lines, each with 2-4 wildcards, depending on the line matched and maybe in the worst case, no way to check the wildcards themselves to 'guess' at which ones came from what line.

Example:

cloves
shield of cloves
10 torches
quiver, with 10 arrows

All these 'could' concievably exist in the same inventory, they all require radically different wild cards to match and it is possible that another item called just 'shield' could end up in there. Exactly which out of the 9 wildcards returned by:

((shield) of (.*)|(\d+) (.*)|(.*), with (\d+) (.*)|(.*))*

would each of the resulting 9 wildcards (counting an item called shield in there) belongs to which line? There is no way to tell without paring the entire set of lines in the script all over again, which completely defeats the purpose of using wildcards in the first place to try to seperate them out. (though this does suggest a minor flaw in my opt function, it needs to jump to the lp line, instead of continuing with the next segment...) Now it may be stupid to even use such a wildcard system in the first place, but mine would do it right, since it doesn't wait to find every single line before passing the wildcards on to be dealt with by a script or anything else you may use them for. My concept also doesn't have the line coloring or omit from output issues.

OK, so you want to break up a regexp (as a pre-processing step) into mini-regexps, and apply them on the fly to lines as they arrive? Ingenious, certainly, but I'm not sure the boundary will always be clear.

I don't see how it solves the colouring (etc.) problems, because you still don't know until you get the last line if the whole thing will match or not.

eg.

You have
.*
.*
.*
in your inventory.

Now until we get "in your inventory" we don't know if the expression is going to match, and thus we can't start colouring the first line. Meanwhile, another trigger might have omitted it.

Also, the regexp parser has things like recursion, look-back, negative assertions etc., that don't fit neatly into the "let's decide to move on a line" concept.

Hmm.. Yeah. On second thought you are right. It doesn't exactly fix the issue with those things.

As for recursion.. If it is recursion within a line, then that wouldn't require spliting, same with negative assertions. The division step is for dividing them into logical groups that can be parsed seperately. I doubt recursion would occur across lines, or maybe I just don't know what you mean by that in this case. Look-backs... Is an issue (maybe), but its one of those things that again, doesn't exactly lend itself to non-static information. But you are right, one reason I am reluctant to even attempt to design the code to break one into pieces is because I am afraid I would miss some odd, but critical instance where it needs to be divided, but doesn't happen. I have trouble everytime I need a binary search and can't find an old copy of one I created previously, this is a lot more complicated. lol

I think I see the difference in what we are trying to do.

You are designing a "state machine" - effectively like the existing system of multiple triggers (say for doing a stat roller, or catching an inventory), where you have something like:

state A
state B
state C
done

Thus, your idea is that this can be expressed along these lines:

state A$state B$state C

The trigger system would break that up into 3 "states" (effectively one trigger that transitions from A to B to C when necessary) and gradually matches incoming text.

Basically like you currently do with multiple triggers that enable and disable each other.

This is fine, and probably workable - although there are still lingering issues about whether or not you colour line A, however in your system once state A matches you probably go ahead and colour, omit, or whatever you plan to do.

My system is more of "take a batch of lines as a block and apply a regular expression to it to see if it matches".

I think my system is conceivably more flexible for obscure cases (eg. if the "overall match" depends on whether line C arrives or not), however possibly slower as it has to try rematching every line. Mind you, the regexp system should quickly exit if it can't find a match on the start of the block.

Assuming I am right about this, I still like the idea of the new system (especially as it is now coded). It makes minimal changes to the existing behaviour, and does give you an easy way of doing things like stat rollers.

Any problems with speed can be managed by simply disabling the multi-line trigger until you think it is needed. For instance, keep it disabled until you do an inventory list, or disable it after stats are rolled, or whatever.

I haven't actually attempted a recursive regexp, but if you read Regular expressions supported by PCRE they give examples of using recursion to match indefinitely nested brackets (something that probably won't happen on a MUD).

Yep. Basically what I mean. I don't really agree entirely with your definition of yours "It makes minimal changes to the existing behaviour". I think that it breaks that behaviour in that you have to give it more information than normal triggers need to successfully process the same lines. The fact that it is also, due to this same issue, not directly translatable from other clients or systems that use such triggers, but don't need the extra "make sure you eat all your peas" step of telling it how many lines to expect, I see as a minor annoyance at best and a complete pain in some nastier cases. For example, a trigger that could concievably match output from another prior list if it looks 'too far' back. Ok, this probably means the original trigger was really badly designed, but it could happen in cases where you have 5-6 variable length lists, you are only capturing the actual items lines in the trigger, there is no ovbvious "begin with this line" part to the trigger and you end up needing to test between 3 to 100 lines in the last list.

I think both our designs probably do have obscure cases like this which will make them fail miserably. Or at least I know there are ones your will fail at and suspect mine may have a few. I don't know quite enough about what could go wrong with regexp to be sure in my case. Maybe at some point I can build such a state machine that works and we can see which one blows up. lol For now.. I suppose yours does well enough, though its one flaw is that most of the cases people have asked about multiline are ones you "want" to leave turned on. The fact that these are 2-3 lines total needed to match would likely to not be too far off what my own design would cause in added overhead, so.... I still don't like it, but for now, I guess I can live with it. lol

Hmm. To tell you the truth.. Recursion in regexp scares me. lol I tend to agree, it doesn't look like something you can expect to have pop up a lot on muds either.

Yes, I found that, but using \z at the end of the regexp "anchors" it to the tail-end of the text block. Thus it will always find the most recent.

Well, single line triggers by definition don't need to be told the number "1", however multiple line triggers need to be told just how "multiple" they are. I think that is reasonably natural.

Say if someone tells you to buy "a box of matches" you know to buy one box, however if they tell you to buy "many" boxes of matches you might reply "how many?".

True enough. This is really only true in rare cases where they don't specify a specific number. Your example is one case where a user may want more than a predetermined number of lines. This however is a rare case. In such cases, limiting it makes sense and is 100% necessary. However, in cases where there is an explicit and undeniable number of lines pre-defined by the triggers match text, you still require the user to specify the exact number of lines to look in to find it. That is what I mean by broken, since its should already know in those cases how many to match and they user can actually cause the perfectly functional trigger to fail completely, just by inadvertently specifying one line less than it would have match on its own. It is a minor flaw, but definitely a flaw.

Well, if I have to count \n characters for you I need to know *whether* to count them or not. In this example it will work:

You see X \n You see Y

That gives me 2 lines (because of one \n).

However this will not work:

You see X (.*\n)* You see Y

Because the \n is inside a repeat group the counting will be wrong. Now to know that I have to parse the regular expression, which could be harder than it looks, and even then it doesn't tell me how many lines to insert, I just discover that it *isn't* 2.

Or this case:

You see (?s)(.*) kobolds

This doesn't even have a \n in it. However the (?s) tells it to include newlines in the "dot" character.

Or what about this:

You see [^a]* here.

According to the specs [^a] *will* match newlines (it matches everything except "a" *including* newlines).

How do you plan to work around that? Have another check box?

Count lines for me? [X]
If not, how many lines: [ 15 ]

Seems a bit silly. I've added another question, and I still need you to specify a line count for those other cases.

Heh. I said it was a flaw, I didn't imply that I had a clue how to fix it, short of not doing it that way in the first place. lol I know why you need it, I just consider it an annoyance you have to live with for your method.

With all due respect, Shadowfyr, I think that you're trying to solve a problem that isn't really solvable.



Let's think about this for a bit.

Regular expressions are implemented as deterministic finite-state automata. Basically, these machines take an input string, and if they are in an 'accept' state at the end of input, then the string is "valid", otherwise the string is "invalid".

A theoretical note:
Any regex can be converted to a DFA, and any DFA can be converted to a regex.


So, with this in mind, let's think about parsing regular expressions "on the fly". This is definitely possible, in a sense, if you run all of your machines in parallel, stopping each one as soon as you hit an invalid input. You will end up, in the end, with one or more success machines, or none at all.

The problem is that a regular expression matching DFA - if it is actually correct - *cannot* know if something matches until all of the input is consumed, except in very special circumstances.

The problem is that we're not just checking for belonging to the language defined by the regular expression. We are also checking for *how far* we can match until the string stops belonging to the language.

So, how could one possibly match "on the fly"? You would take input as you receive it, and then feed it to your state machine; in the meantime, what do you do with the state machine? What do you do with the MUD output? Do you leave the machines hanging as you receive and print output, and maybe later act on the input - and perhaps go change the output window contents?


I think the major problem that you are grappling with is that yes, the problem *should* be simple, but it just isn't. If the PCRE folks haven't done this, don't you think there is a reason? The solutions that Nick proposes add a degree of complexity (which I consider to be fairly minimal) but provide correctness. The solutions you provide seem to be "practical" in some cases (but limiting in others), but also just seem to be incorrect. Yes, they would work in 95% of the cases. But in those extra 5%, you'll have to apply some kind of fudging that just really isn't a good idea.

As far as I can tell, Nick's solution can do everything yours can. It involves a little extra work in some cases, but provides functionality that your method does not. Personally, I don't think that the list matching you dismiss is that useless. In any case, I think that the added complexity (which again I don't think is that major) is worth the generality and especially correctness. To be frank it seems that your main problem is just that it's not what you had in mind for certain specific examples. :)

It wasn't what I had in mind for most examples. You are correct that his does automatically handle the case of 'did all of this match'. With mine you would need to keep track of the data yourself, until it either finally succeded or failed. In some respects I think this tends to be more flexible. I didn't really consider the issue with needing to know 'all' of the lines before doing something. Unfortunately, the very things I was looking for a way to fix, can't be fixed by either method, i.e. coloring lines or omitting from output. For those you end up being forced to fall back on old methods. Neither concept really solves this or necessarilly can.

The only real advantage is that mine didn't need the extra 'look for x number of lines', though it would have required doing what Nick is doing to work correctly (as you point out), i.e. storing each line matched, until a final match happens, then doing the stuff in 'send' or a script. Its not a big change to the design and would, with that change, work exactly like his, but without the risk of catastrophic malfunctions in those cases where the user does something dumb, like sending two 'inventory' commands in a row and using an insane value like 50 for the number or possible lines to match. It isn't an impossibility that such would happen or that the inventory would change between them, think, checking it, then having some clown into the room and hand you something... Ooops! Its this possibility for error that I don't really like about it, even if I was admittedly slightly off track on my own design.

Oh, and mine wouldn't need to be turned off when you 'think' you don't need them.

Well, I'm not sure the problem you bring up is actually a problem. It depends on whether the regular expression is greedy or not. You would have to make it non-greedy in this case. And if the user is writing bad triggers, well, that's the user's fault. :-)

I think that perhaps there could be a more official kind of support for stacks of single-line triggers. That seems to be your method of choice; having triggers that turn each other on in some way where the user keeps track of data. Perhaps you should formalize exactly and precisely what you would want to see such a system do, instead of lots of ideas... The thing is that Nick's system is an additional feature, the subsequent triggers are more of a way of using the current system, if you see what I mean. What more exactly do you want added to that?

Again, you anchor the trigger with \z as the last character so it always matches the most recent inventory. However I agree that you may need to use ungreedy wildcards to ensure you get the smaller rather than the larger inventory. However this is always an issue with regexps.

Yeah Nick. Already forgot that \z bit. lol

And Ksilyan, like I said, I failed to consider the way multilines need to match. If I had, I would have suggested returning the entire thing in a wildcard, just like Nick is doing. In that case, the *only* difference between my state machine and Nick's system is that mine wouldn't need to be told how many lines are needed before hand. The bahaviour would be basically identical in all respects. I am not sure exactly what you think I need to 'formalize' anything. I wasn't thinking carefully enough when suggesting that such a state machine based system should return each line and group of wildcards immediately. But now that I am thinking in those terms, it could have done both with the addition of one simple switch, which could have told it to treat each match as a seperate event when you needed that. As a rule, it isn't, as you point out, how multiline triggers normally work, so making the default behaviour return the result 'in total' would make more sense.

Given this, the real difference between the approaches is mine providing one option that Nick's can't, and the fact that his executes 'after' the lines are all available, while mine would process each individual line one after the other and return the result as soon as all criteria have been met. Mine would have a slight added overhead for all instances, the overhead for his grows for each line it needs to test against. But *both* could work exactly the same way with respect to 'how' and 'when' they return results. I just goofed up when initially thinking about them and what 'multiline' really means.

How do you know when to stop matching?

Umm.. When the termination string you put into the trigger tells it to or it fails to match something? lol Seriously, if someone makes one that is so sloppy it matches forever, why is it the fault of the algorythm or client that they messed up? Even Nick appears to be using a bailout (according to one post I read today) of about 100 lines, so the buffer can't be set to like 4000 lines. If the trigger is designed right, it should have an obvious point to stop, if not, telling to to look for 20 lines is convenient to have, maybe, but all your doing is specifically telling it to bail out of the matching 'before' whatever internal limit the client itself uses. But having it optional, when you need it is significantly different than using the wrong number and having to buffer 100 lines, when you actually only match 10, or worse, not matching 11, because you told it the bailout should be 10.

Consider this:



Now consider the regular expression:
^Inventory:$(.*)^$

Or something to that effect. How do you know when to stop matching? Do you get the first bit? Or the whole thing?

In this case it's clear, the answer is that you only want the first. But that means your algorithm is non-greedy. What if you want it to be greedy?

Perl has a non-greedy specifier, ?, which could work. But Nick already mentioned that as the solution.

How would you propose, Shadowfyr, to deal with greedy vs. non-greedy regular expressions?

I mean no offense but it seems that you are operating under the "it's obvious what I meant" principle. As you are well aware, you can't design a system based on what the user meant; you can *only* design systems that do *exactly* what the user specifies. Otherwise you are sacrificing correctness.

This is pretty-much my point. That particular hypothetical pattern is matching:

Inventory:
(anything at all)
(a blank line)

How does the state engine know to move from (anything at all) to (a blank line) given that (a blank line) falls into the category of (anything at all)?

This is why the regexp executer really needs to take the text as a whole block, you can't really break it into smaller pieces and have an external decision about when to move from one piece to another.

Even having a simpler expression:

Inventory:
(anything)
You have x items.

... still has that problem. The phrase "You have x items." still matches the middle expression of (anything).

I have conceeded tha tmy idea does not work with greedy expressions. I did suggest that in those cases it is just as easy to allow the user to specify an 'optional' limit on the number of lines. Or did you miss reading this? You seem to still be operating on the assumption of the original idea, without taking into acount the changes I proposed to make it work more like Nick's. The point here is 'optional' *with* 'greedy triggers', not 'always required'. Nick's always requires it, even when the regular expression itself has no requirement for this. And as I said, it is a difference between a *slight* increased overhead for all regexp and a *major* increase in overhead the moment you start looking for large numbers of lines, since Nick's has to retest the entire buffer of lines it builds, every single time. Mine only tests one line at a time, but *could* be made to do:

1. Non-greedy - Store all matched lines until all are found or it fails.
2. Greedy with a user defined limit - Store all matched lines, but fail if the limit is exceeded.
3. Act like each match is a seperate event and return each line + wildcards as they are matched, instead of storing the entire result.

Nick's does:

1. Assume all triggers are greedy, require a limit to be set and store everything until all matches succeed or one fails. It also has a built in limit, (I assume from another post I saw), so even if the user specifies 200 lines, it cannot match, since the internal limit of the buffer is 100. Unless I misread the meaning of that post where you asked how many lines users are likely to want to test Nick?

Mine is more flexible because it doesn't need the buffer, but can do exactly the same behaviour, when needed.

However, the point is mute anyway, since I don't have the skill with parsers to build one that can correctly delegate each section of the expression to the state machine instructions. Thus, unless someone else built and proved the concept, it isn't going to happen.

My point is that I don't believe such a parser is even possible to build, to be precise and correct at the same time.


If you're storing all matched lines, you're not really being non-greedy. A non-greedy expression would match the *least* amount of lines possible.

This is greedy, yes. Although there is the problem of matching lines that may not have the terminating condition - imagine you have a list that is 105 lines long, but your limit is 100. When you reach 100, what do you do? How do you *know* that the text beyond those 100 lines still matches the regular expression? Perhaps everything you have "matched" up to now simply does not correspond to the regular expression.

One cannot determine wildcard matches until one has the whole match, because the wildcards are built during the parsing of the whole text.
How does one "return" lines and wildcards as they are matched? Again, you have the problem of terminating conditions.

Basically, like I said I have the feeling that you're operating under what is convenient to use, but not what is correct in all cases. I'm truly not convinced that it's possible to write a parser as you are suggesting, that can somehow "guess" if input beyond what it received is valid or not.

Huh?? If I have it match three specific lines and *only* those lines, then obviously it is going to return "all matched lines". Your arguing symantics here. It does match the least amount possible, but it returns all that it matched, not all that it *could* match.



Please, enlighten me how providing a user defined limit or having it automatically fail when it hits 200 lines (check Nick's release notes, this is what his does) differs from Nick's?

If it is 105 lines long and you tell Nick's version to look for 100 lines, it still isn't going to find 105 lines. Nick's answer *and* mine is that the limit means that it is *a limit*. If by that point it has captured all the lines it was asked to, in cases like "(?s)^You have\:$(.*)*" then it is considered a match, it can't look for 105 lines, because you told it to "only" find 100. If the lines is "(?s)^You have\:$(.*)*^$", then the rules change. In this case it would fail if you have 105 lines and return nothing, again because you told it to stop at 100 lines. This is no different if you use mine or Nick's, but mine only *requires* it in the first case, since it can correctly match the second case, even if you have 200, 500, or 1,000,000 lines. It already knows 'when' to stop in such a case.



This is a special case, it is not intended to act like a normal multiline trigger, but to provide a single trigger to solve a problem that currently requires several triggers working together to work. Again, this is *not* the normal behaviour it would have, but an option that could be turned on in cases where such behaviour is wanted. Yes, it doesn't technically work the way multiline triggers are supposed to, but it isn't supposed to in this case and it would be off by default.

I'm not sure I see what you're talking about. The problem of greedy vs. non-greedy comes when you have wildcards in your regular expression. So it wouldn't be matching three specific lines; it's matching a block of text.

I was just commenting on the fact that your statement wasn't really precise in an algorithmic sense.



I misread your original statement, I did not see the part about failing upon hitting the limit.



It seems to me as if you're breaking the elegance of the system for the sake of convenience. It just reeks of "hack" to me. :)

There is a built-in limit, yes, basically to limit the number of lines that are stored "just in case" you may want to do a multi-line trigger. After all, each line takes memory.

However you will not be allowed to ask to match more than the number of lines in this limit. It has been increased from 100 to 200 in case some people have big inventories, or "who" lists or whatever.

Ah, true, but not more inelegant or a hack than dumping a copy of every line into a bucket and telling your assistant to dig through it after every line you drop in to see if now contains everything you wanted to find. ;) lol

But if you think about it, that's the only way for the regular expression to behave coherently. You can't have a wildcard for every line in the list, simply because you don't know how many lines there are.

Again let's consider:

^Inventory:$(.*?)^$

This time note that I have the non-greedy specifier. This regular expression will match on "Inventory:" followed by a newline followed by anything followed by a blank line.

How are you going to express the notion of making a new wildcard for every line?

It seems to me that the cleanest solution is to just get the set of lines matched, split them by newline, and then do whatever you need to do with each line.

Furthermore, let's assume you wanted to have a sword in your inventory. You could check that like so:

^Inventory:$(.*?)sword(.*?)^$

So I'm not sure what the problem is. You say that we have to go looking in the bucket for what we were looking for... well... let's be more precise, what were we looking for and we can't we find it with the current method?

I'd imagine that a scripting function could be added that given a block of text, returns an array of each line. Basically a split function, that operates on newlines. Would that solve your problem?

Geeze.. I think you are still missing the point. lol Both our methods can do the same thing, mine just doesn't need the bucket. Having a script function that returns an array? Umm. Try 'split'... Anyway, I just said I thought taking extra time to create and search a buffer, that mine doesn't need, is inelegant and a hack. This doesn't mean it doesn't work. Inelegant means clumsy and excessive when another method could be more efficient, hack in this case means "the first thing that came to mind", rather than the most efficient solution. I can't see how forcing the user to define a buffer to search, even when the trigger is absolutely and undeniably not greedy, can ever be called efficient or elegant, therefore at the very least *some* other method must be better, even if you don't agree it is mine.

The only thing that mine does that is a hack is the third option for immediately acting on each matched line. Some goofy thing that generates an array from a bloc kof lines is still a hack, since you are not handling the lines as they arrive, in fact it is what you end up doing if you want to do anything useful with Nick's output anyway. No, it doesn't solve any problem I see with the design.

It isn't about "if" they can both do the same things, it is about how they do them and the overhead they introduce that can slow down operation of what, up to this point, has been a client that strives to avoid things that can seriously slow it down.

But again, this is a mute point. I doubt I could successfully write a pre-parser and without a working version of mine, Nick's can and will get the job done. My personal opinion is simply that we are paying a higher price for the behavior than we might have if my system could have been developed instead.

Well, fine, the point may be moot. Nonetheless, I still think that you are trying to do something that just can't be correct in all cases. The day you bring me a working program or algorithm, I'll be convinced; until then let us agree to disagree on the feasibility of your project. :)

Yes, perhaps time to stop discussing it. :)

I don't think my method is perfect, however it was quick and easy to do - easier than some sort of on-the-fly state machine, and the easy solution has the advantage of simplicity, and simplicity has the advantage that it is likely to have less bugs.

For those that recall the many, many, discussions we had in the past about multi-line triggers and why they are hard to do, this forum thread confirms that it is a problem that doesn't have a quick, simple solution.

It reminds me of the saying:



My solution, which is now available for playing with, may not be the neatest in the world, but if it works to easily process stat rolls, or player stats, or inventory lists, or multi-line room descriptions, then I will be happy. :)