When is DeleteLines() fully processed?

I notice that after calling DeleteLines, GetLinesInBufferCount has not changed. The lines that have been deleted are still valid indexes into the buffer, but are blank strings.

This causes havoc for trying to clean up after a script that has deleted lines - for instance deleting surrounding blank lines.

How can I call DeleteLines(), then at some point later in the same alias / trigger, get an up-to-date view of the buffer?

As I said in another thread, the client always has a "current" line at the end of the buffer, that is, the buffer is always non-empty, and it always has a current line to which incoming text is appended.

So DeleteLines (1) would delete the current line, and then insert this "empty line" at the end, so the number of lines in the buffer would not change.

If you test something like this:



The number of lines displayed will go down by 4 each time (because of that added line). So the number is being updated, it just isn't quite what you think it is.

Each line has a unique time-stamp ( GetLineInfo (x, 13) ) so you may be able to work out from that if this is an "old" line or one you put there yourself.

When the buffer looks like:

and I call DeleteLines(2), the buffer then looks like:

Which is what I would expect. I note that my prompt sting has a CR on the end.

If the buffer look like THIS:

and I call DeleteLines(1), the buffer remains the same:

but if I call DeleteLines(2), the buffer becomes:

At this point, if I scroll up (or down), the buffer becomes paused, and will not scroll back down to the blank line, nor will it unpause until more data comes in from the MUD and you scroll down.

So while prompt lines can acheive the desired result, there's some consistency problems between prompt and non-prompt lines. Is there a way to treat all lines like prompt lines, maybe?

Or is there just no way to delete a single blank line on the end of a buffer without inducing a scrolling bug?

What do you mean "becomes paused"? Does the word MORE appear in inverse on the status line?

Look, adding data to the output buffer (a doubly-linked list) was one of the first things I did in the client, after changing from a fixed 1000-line array.

As such, it is probably the most badly implemented part. I mean, it's reliable normally, but fiddling with it can have strange consequences. A while back I tried to change from using the Microsoft CTypedPtrList class (with its strange behaviour of the way you "walk" the list) into an STL dequeue type (which would have been better in many ways).

In doing that all sorts of odd things happened, for example the last line wouldn't be drawn, or it would crash, or some damn thing, so I went back to the older method.

One of the worst design decisions I made, I think, was having that "existing last line" thing, which is causing all your grief. For one thing, if the last line type is "output" and you type something it has to change its type from "output" to "user typing".

However having acknowledged that it's flaky, it generally works, unless you do something unusual, which --- dare to say --- doing DeleteLines is.

It isn't helped by the weird way Microsoft implemented lists, unlike the STL way, where to get the next element the pointer actually advances to the next one plus one. Ditto for getting the previous element.

If you can spot an obvious bug in the C++ code, fine, I'm happy to fix it. But I'm worried that fiddling with it will introduce more bugs than it fixes.

As a work-around, I suggest omitting all blank lines. I have a plugin that does just that.

And/or make a plugin that omits duplicate prompts (ie. it omits all prompts, and puts one back if it is different from the previous one).

This was in 1995, BTW, about 18 years ago. For a full history see:

http://www.gammon.com.au/relnotes

Yes, the word MORE appears, and the 'paws' are depressed. I have since discovered that this bug only happens when i call DeleteLines using the script prefix on the input line, but the remainder of the problem... remains.

As to the nature of your buffer, it's a shame. Should I find the time to investigate the source, I will do so, but as you mentioned, I'd be opening a can of worms.

Deleting ALL blank lines is not an option. It's hard to read, ugly, and the server already offers that option if a user wants it. And it's trivial to accomplish if I had wanted to.

And prompt deduplication? I do that already. This bug is related to keeping the prompt as the LAST line in the window, not a blank line. I could try omitting and pasting it back. I'll let you know how that goes.

I'm 99% to where I want to be. Full ANSI line move/copy with blank line and prompt deduplication. I guess I'll have to live with the 1 little bug that's left.

And with that little test, the answer:
calling DeleteLines(3), then AnsiNote(prompt) still produces the extra line after the prompt.

I know I shouldn't have to suggest work-arounds for buggy code, but here goes ...

First, omit all blank lines. Then have an algorithm that notes a blank line when certain conditions are satisfied (eg. before a prompt). The algorithm would remember that it had just done that, and wouldn't add two in a row.

Or even this ...

Match on blank lines (like the omit blank lines plugin does). Omit them (in the trigger). However under certain circumstances, put one back. That way you aren't calling DeleteLines.

The first strategy is no good, because I can't predict where the server will place blank lines, and I don't want to reconstitute information that is already available to me (i.e.: that the server inserted a blank line)

The circumstance for inserting a blank line in the second strategy is:

1. A blank line was last sent (track from omit)
2. A nonblank line has arrived.

This will require triggering on all nonblank lines and then inserting a line before the triggered line. How could this be accomplished?

Alternately, you could omit ALL lines and AnsiNote them back, with blank lines inserted when appropriate. Can you still scrape style information from a line that was omitted from the buffer?

edit: Evidently not. Just tried it.

You should be able in send-to-script, not send-to-script-after-omit.



Er, yes, but if you get the line count (line from server count) then you should be able to detect that this is a singleton blank line.

Something like this:

• Omit all blank lines
• In the trigger processing, see if we just omitted a blank line previously (based on line-count-from-server)
• If not, note a blank line (ie. put it back)

This seems to work (in Lua, sorry):



That is omitting all blank lines, but if the previous line was not blank, then it puts the blank line back.

That solution does not solve my problem, as it still leaves the final line blank EXCEPT when the last line from the server is a prompt.

So unless I find I way to make other lines like prompt lines, or I append a blank line AFTER a prompt from the server, there will always be an inconsistency with the last line.

What's the problem exactly? The last blank line? It's always blank isn't it? Can you post a screenshot?

This is how the buffer looks when a prompt comes in:

http://imgur.com/rRECH0y

If any other line comes in, there's no way to return the buffer to that state.

Yes, but that's by design. A prompt line is (usually) not terminated by a newline, so you don't see the extra (blank) line.

Once other data arrives, it is terminated by a newline, and thus you see the blank line at the bottom of the buffer, which the start of the next line. In other words, it has shown the newline.

I think I see what you are saying. If you get a prompt, it will sit at the bottom (as there is no newline). Then if some blank lines arrive the prompt jumps up one line.

But you said you wanted to see blank lines from the server. If you omit all of them, the problem goes away. My suggested trigger omits all but one, but as requested, you see at least one.

Your suggestion does not meet my design goals, which are (in no particular order):

1. If the final line is a prompt, it should be on the bottom line, as the server intends.

2. There should be no consecutive blank lines.

3. Certain lines should be moved to a different world.

I have this completely functional already, except for one case: when all lines that have been received after a prompt have been moved, effectively leaving the prompt as the final line, the prompt is not on the bottom.

So your trigger does not meet design goal #1

OK, but:



You want to see server blank lines, you just said so.

So if the prompt is followed by a blank line, then you reasonably should want to see it.



In this particular case, the server also intended a blank line, otherwise you wouldn't have this issue.

Let me guess that what is happening is that you have a prompt line, then some chats arrive which are moved to a different window. However a blank line after the chat now makes the prompt "jump" up a line, is that right?

Can you explain why you're using DeleteLines in the first place? I don't understand why you would want or need to.

This is how I block trailing blank lines after other blocked output. Any time you "move" a line of output, enable this trigger group.

^^^ Yes, that can be used to delete trailing blank lines, but not leading blank lines, as would be the ones between the moved line and the prompt.



I don't think I ever denied wanting to see blank lines, except in one case.



No need to guess, it's exact problem I've been describing!

When a prompt comes from the server, there is no blank line following it.

When subsequent lines come from the server, it sends a blank line followed by the subsequent lines. If I remove said lines from the buffer, the output buffer should be effectively unchanged, i.e.: the prompt is still on the bottom line.



I'm using DeleteLines because the built-in trigger system is not flexible enough for my needs.

1) Triggers cannot delete themselves.

2) There is a bug where a one-shot trigger will fire twice if it is set to keep_evaluating under certain circumstances (which I have not fully determined).

1 + 2 = There is no reliable way to make a one-shot trigger that is keep_evaluating.

3) It is difficult to do substitution of information and keep processing. For instance, one trigger changing the data that subsequent triggers process WITHOUT changing the line itself, i.e.: transparent to the user.

You might decide to store the interstitial data in a variable for the later trigger, but what if the substitute trigger is not present? You will have created a strong dependency between the two triggers. I am trying to design something modular, loosely bound.

4) There is no way to conditionally stop trigger evaluation (that I am aware of).

Well, I suppose you could, but you would have to go disable every other applicable trigger and then re-enable them at a later point. Which means you need to know about them all and have a reliable method of re-enabling them. So again, strong binding, if this strategy would even work.

5) Omit does not work for situations where the trigger represents the final line of complex data output.

Let's say you want to capture all room information. You know that room titles are proper case, the desc ends with [Exits: .*], and there is an item list (might be nothing) and a mob list (also might be nothing) that follows, terminated with a prompt.

How do you write a trigger that copies all of these lines and omits them? A multi-line trigger might work, but what if a chat line appears between the mob list and the prompt, and that chat line is not deleted? Maybe don't trigger on the prompt, but try to include the null items and mobs, while somehow not capturing other things? That's going to be a pretty complex regular expression, which would be a pain to a) create, and b) debug.

Procedurally analyzing the output and selectively moving the lines is the simple approach. Find a prompt. Analyze the output buffer for a room title. Read forward, moving the title, the desc, the exits, the item list, the mob list. Done. If there were any chat lines in there, they remain.

Therefore I have a single trigger, (^.*$), that captures all lines from the server and performs its own pattern matching and processing.

It does grant me any processing I wish though, including:

1. On-the-wire substitution (synchronous)
2. Conditional/dynamic line omission
3. Conditional/dynamic keep evaluating / stop evaluation

It is a shame that the behaviour of DeleteLines is not fully documented or intuitive, but the alternative is wading through a mire of other problems and things I cannot control.

There is something odd about the output buffer behaviour, I'm looking into it.

Specifically, if you receive a line from the MUD, terminated by a newline, the extra blank line added by the code (as soon as it sees a newline) is not displayed.

However something (I'm not sure what, but deleting the last line seems to do it) makes the last blank line be displayed.

So, it's more of a display issue.

For example, typing this into the command window (with "/" as the scripting prefix, and Lua as the language):



The last line is shown as length zero (as expected, as it adds a new empty line) however I am seeing the last line from the MUD at the bottom, not this blank line.

---

Try this:



That deletes the last line, and then scrolls down to the end of the buffer, invoking whatever logic that exists to hide that last blank line.

Those two commands invoke the pause mode bug.

This is not actually true. They just can't delete themselves without a very small delay.


If you trigger on the prompt already anyway, you can use my suggestion above to delete the blank line following it.

I have not had this bug reported before.



I can't reproduce that. It just seems to work for me.

It seems to me that you are fighting the client here. Having a single trigger, and doing it all in there, is throwing away all the trigger sequencing, omitting, scripting, etc. that are designed into them.

Exactly. Because, as I said above:



I have designed my own sequencing and omission. It works better than the built-in one (excepting this thread topic). A key feature missing from the built-in system:

Conditional, dynamic keep / stop evaluating

I any trigger processing code can signal trigger processing to stop for the given line. How could I do this in MUSHclient's triggers?

For instance: You cannot see!

You have two trigger handlers, one of which casts 'cure blindness' and one of which eats a pill of cure blindness.

How do you write these two triggers so that they remain independent (no strong binding) but only one will fire if its conditions are met? You obviously would not cast if the room were hushed or you were out of mana, you would eat the pill. Likewise, you wouldn't eat the pill if you could spellcast.

Now for the solution you're thinking of, extend it to also support wearing an earring of cure blindness, or applying a salve of cure blindness, or any number of further actions.

None of the code can reference the other triggers, because the software is modular, and the user might not have any of those items (or want to use them, or have the salve use module loaded, or...), or the ability to cast that spell.

It's ridiculous to do this without conditional trigger termination.

I also mentioned 2 other things that MUSHclient triggers don't do. So yes, I am throwing away the built-in trigger system.



From my experience, if you have two matching triggers at the same priority, both of them keep_evaluating and one of them one_shot, the one_shot trigger will fire twice. Again, I have not fully discovered the conditions where this is true. I recall that the non-oneshot was in a plugin while the one-shot was not.



Let me rephrase that, then: triggers cannot delete themselves synchronously. It's inelegant to set a timer to delete a trigger, and opens a window for a race condition. In my system, I can delete any (internal) trigger at any time with no ill effect.



Okay, I think you took what I said out of context. I was talking about the room capture scenario, not the blank line omission. If you want to respond to the room capture scenario, please flesh out what you're saying, because it doesn't make sense here.

And, as I mentioned, your solution does NOT solve the thread topic problem. I'm trying to delete a leading blank line, not a trailing one. The trailing lines delete fine with a single call to DeleteLines()

As soon as I execute that command, the 'Paws' Button depresses, and it says MORE at the bottom. This only occurs after I have connected to a server.

Here is a video of it occurring:

http://youtu.be/dJ1syii7NLs

Check (set) the output option "unpause on send" and that problem seems to go away.

I'll look into what that is about.

OK, I found the "paws" problem. When a line (or lines) were deleted it was not updating the scrollbar information (ie. there are now less lines). This had the side-effect of making the scroll bar thumb move up a line. That caused it to go into "paws" mode, and if you didn't have "unpause on send" it stayed paused.

So now, doing this seems to correctly delete the last line:

That video was taken with the option "Unpause on send" already checked.

Actually you shouldn't have to do the "end" thing.

I've redone it to call "addedstuff" which adjusts scroll bars etc. and has the side-effect of scrolling to the bottom, if not paused. So this should work fine.

So now, doing DeleteLines (1) should correctly delete the last line, and scroll so that the previous line is now at the bottom.

Fixed in version 4.89.

https://github.com/nickgammon/mushclient/commit/58e00450d071

If the triggers are in the same plugin (or in the main world file) then when one matches it stops further matches (unless keep evaluating is set) which is intended to let the "higher priority" match take effect.

For plugins, plugins are independent, by design. Even if a trigger fires in one plugin, the others are still handled. The intention is that (say) if blindness is detected by one plugin, it doesn't stop another plugin, perhaps by a different author, of handling it. Also the plugin evaluation order is not strongly defined, so you cannot really rely on plugin evaluation order.

You found a bug. Thanks for the bug report. It's been fixed. It looks like some of my explanations about the "last blank line" were (whilst correct) not the actual problem.

There was already a test in the code to not show that last, empty, line, so that was not the real problem.

Awesome, thanks for the quick fix!

re: Triggers: Exactly. A trigger cannot dynamically cause trigger processing to stop. It either continues processing or not depending on whether keep_evaluating is true.

In the system I use, I can return True from any trigger handling code, denoting 'terminality', and no further handlers will be called.

So in the blindness case, casting gets assigned a higher priority, then if it is able to cure the blindness, it returns True, and the pill, salve, etc. triggers will not be processed.

This is much simpler than disabling triggers/groups and re-enabling them later. It's also more efficient and reliable.

I can see some sense to that, but in your case you are using a single match-all trigger anyway, so it wouldn't make any difference if this was added?

You asked why I was using DeleteLines(), I responded that it was because the trigger system was inadequate, and this was an example of how.

I'm not asking you to add that feature. It was merely part of my explanation. :)

Thank you again for the quick bug fix. How often do you update the binaries?

I usually do it the same day a major bug (such as this one) is uncovered. However I'm exploring the idea of allowing triggers to stop evaluation, to save doing two releases when one would do.

Version 4.89 now released:

http://www.gammon.com.au/forum/?id=11950

Also added the stuff about stopping trigger evaluation, in case someone found it useful.