Question about implementing MCCP

Hey guys,

I have been engaged in the creation of a mud engine from scratch. I started with tinymudserver v2. The project is reasonably far along now.

My question though is about implementing MCCP. I recently decided to investigate adding support for this protocol but unlike other protocols (MXP and MSP) there are no really easy to use guides to implementing them. I don't really want to have to reverse engineer my solution from other mud's snippets too.

As far as I can tell, all you have to do (from the MUD's perspective) is the following:

Send IAC WILL COMPRESS and IAC WILL COMPRESS2, and listen for the response (IAC DO COMPRESS / IAC DO COMPRESS2 or IAC DONT COMPRESS / IAC DONT COMPRESS2).

Once we have this response, we prepend all compress output to the user with:

IAC SB COMPRESS SE or IAC SB COMPRESS2 SE.

Then we run the zlib function "deflate" on the data, send that to the user, followed by Z_STREAM_END.

Is that it? Am I missing something, or doing something above incorrectly? Note that I haven't actually attempted to do the above yet - I'm a firm believer in having a design first before implementing something so this is part of my research and design phase.

As best as I can infer from reading other sources, that's all I need to do. But some of the snippets I've seen to do this stuff are incredibly long and complicate...I wonder whether I'm not leaving something out?

Any help appreciated.
Cheers.

Looks fine to me. I implemented COMPRESS2 a while back on a mud server I wrote (I don't bother with the old COMPRESS, it has been deprecated for years, and supporting it only hurts COMPRESS2) and did things about the same way you did.

Let me know if you have any concrete problems.

Not quite. You are roughly right. You prepend IAC SB COMPRESS SE *once* - after that all output is assumed compressed.

So, once the client agrees to handle it, you send:



You might send Z_STREAM_END when closing the connection perhaps.

The deflate algorithm can handle streaming, so you can stream out as much or as little as you want, it remembers its state, so the other end can handle individual packets of compressed data OK.

I had a backup of the MCCP web page, here it is:

Just in case the link you had before went bad, I still have the page archived here: http://mccp.smaugmuds.org/

Well, you can actually change the compressed state as often as you possibly want by ending the stream and restarting it again. So what he said is entirely correct - it is just a very weird way to use MCCP since virtually every server starts it and never turns it off anymore. :)

Well, it's incorrect in two major ways:

1. The spec says to compress everything after the negotiation. Not to compress, per message. It says it "may be desirable" to terminate compression at some point, however I take that to mean in situations like a hotboot, or if the client requests compression to stop for some reason.
   
2. It defeats most of the point of compression in the first place. Compression works by looking for redundancies in text (which is why you can't successfully compress random data). By stopping and starting the compression, per message, you throw away its "dictionary" of words that are being repeated. See below for more details.

The MCCP protocol uses zLib, which is available in MUSHclient's Lua interface, so it is easy to demonstrate the difference. This script I wrote in the Immediate scripting window:




Results:



Let's assume someone is standing in Darkhaven Square, typing "look" 10 times. The figures show the difference.

The first figure (472) is how much is sent uncompressed, each time. And after 10 times, we have sent 4720 bytes.

The first time, the compressed figure is shorter, certainly - 271 bytes. However after 10 times, we get a big difference:

• Keeping a compressed stream: 311 bytes - only slightly more than typing "look" once.
  
• Stopping and restarting compression: 2710 bytes (271 times 10). Plus we need to add in the IAC SB COMPRESS SE bytes to *each* packet, so it is really 275 * 10, namely 2750 bytes.

So, by doing it the recommended way, you have only sent 11% of the amount of text, after the player only typing "look" 10 times.

I'm just trying to point out that turning compression on and off isn't just another way of achieving the same result, it is much much worse.

Oh, I fully agree with that. It's a stupid thing to do in most (~99.9%) of the cases. I was only saying it is very much possible and not in any way 'physically' required to keep compressing by the spec.

Thanks as usual for your help Nick (and others). The information available at the time was incomplete and confusing, so I'm glad you were there to provide your old advice. I only assumed I had to send Z_STREAM_END because one of the pages mentioned that "Indicated the end of a compressed stream". I had no idea whether that was the whole session or just a block of text. That question has now been answered.

Oh, one thing though... Make sure you flush the stream when you know output won't be immediately following anymore, for example after you echo'd a prompt. Otherwise it can be partially stuck in the decompression buffer for a while leading to an odd playing experience where lines sometimes don't show up or take ages to show up. :)