MUSHclient benchmarks

Where are the comparisons?

This page used to list comparisons between the time taken by 16 different clients to list a large file. However keeping it up-to-date is proving a bit tedious, as it is time-consuming every six months or so, for 16 clients, to

• Find the site where the latest version is
• Download it
• Install it
• Work out how to use it
• Test it

Some clients have released newer versions which are faster than were previously listed on this page, so it is not fair to leave the old figures lying around indefinitely.

Thus, we will just list the time taken by the current version (3.17) of MUSHclient to do various things. You can compare those times to other clients if you like. Of course, running the tests on a faster PC will tend to make any program run faster, so you should really compare your program to MUSHclient on the same PC.

To be fair, most of the leading clients seem to perform very well in terms of speed, with the earlier benchmarks showing that the top ones can display around 4,000 lines of text in between 2 and 15 seconds. In practice, that is fast enough for mudding purposes.

Still the fastest

On the basis of recent re-testing (April 2002) we stick by our claim that MUSHclient is the fastest MUD client (based on the tests described below), however if someone wants to submit the name of a client that they have tested as being faster, then we will withdraw that claim, if we can reproduce it.

Our benchmarks

The tests were conducted on a 450 Mhz Pentium III running Windows 98. This was connected via a LAN to a fast PC running OpenBSD 3.0 as a simple telnet server. The output window was set to 100 columns, 22 rows, displaying Courier New font, 10 point size. In global configuration "smoother scrolling" was enabled. The output buffer was set to 10,000 lines.

Test last run on 17th April 2002. Times are in seconds. In each case, because MUSHclient is so fast, we actually ran the test 5 times and divided the result by 5. The exact tests, with notes, are as follows:

What were the tests?

• Test 1 - raw speed

  Type the file "/etc/termcap" from Caldera Open Linux (ie. "cat /etc/termcap"). This is 6716 lines, 11743 words, 288451 characters (bytes). No triggers or colours. This is a test of raw speed. If you want to reproduce this test, please download that file as there are various versions around of differing sizes.

• Test 2 - colours

  Type the file "high.txt" which is a log of a session with ANSI highlighting in it. This is 5378 lines, 33087 words, 437728 characters (bytes). This has a lot of colour changes in it, so it tests parsing of colour codes.

• Test 3 - colours in a proportional font

  Same as test 2, except the font is now Arial 10 point. This is to test drawing where each character is a different width on the screen.

• Test 4 - Non-matching triggers

  Switch back to Courier New 10 point font, and add the following triggers: "xxxx0000", "aaaa1111", "bbbb2222", "cccc3333", "dddd4444", "eeee5555", "ffff6666", "gggg7777", "hhhh8888", "iiii9999". These are added as regular expressions so they can match anywhere on the line. This is a test of how fast MUSHclient can process and discard triggers that don't match.

• Test 5 - matching triggers

  Still using Courier New, 10 point font, add another 5 triggers, "a", "b", "c", "d", and "e". All regular expressions. These should match most lines in the file. Each one was set to call a script called "trigger" written in VBscript like this:

  
  sub trigger (sName, sLine, wildcards)
  dim i
    i = 1
  end sub
  

  This was to test speed actually matching triggers and calling scripts.

• Test 6 - script execution time

  To test the script interpreter we ran the following script:

  
  for i = 1 to 1000000
    a = i
  next
  

  This ran in 3 seconds. Note that the number of loops executed is 1,000,000, not 10,000.

• Test 7 - MXP parsing

  Type the file "mxplog.txt" which is a log of a session with ANSI colours and extensive MXP tags in it. This is 3611 lines, 28328 words, 338495 characters (bytes). This has a lot of MXP tags in it, so it tests the MXP "engine" at some length. To do this test we set "Use MXP/Pueblo" to "Yes - always", because typing this file did not actually do the telnet negotiation that MUSHclient expects. For this test triggers were disabled, as we wanted to test MXP parsing, rather than trigger evaluation.

Other benchmarks

• zMUD benchmarks
• Previous MUSHclient benchmarks - now out-of-date

If you know of other benchmarks please let us know so we can add them to the list of URLs above.

Submitting a faster client

If you believe another client is faster than MUSHclient, running on the same PC, please email us with your measured times. Don't just write in saying you "think" client X is faster, or "it must be faster" because it is a console application, or is written in a certain way. We are interested in actual, measured results. The information above should be sufficient to reproduce the tests, the files we used are available for downloading. We also want the figures you got for MUSHclient on the same PC, as you can always get faster figures by comparing MUSHclient on a slow PC to another client on a fast PC. Also please supply the URL from which we can download the client to confirm your results.

Comments to Gammon Software support