SMAUG : Running the server : game_loop(), select(), and dual core CPUs

Gammon Software Solutions forum

See www.mushclient.com/spam for dealing with forum spam. Please read the MUSHclient FAQ!

Entire forum
SMAUG
Running the server
game_loop(), select(), and dual core CPUs

Home \| Users \| Search \| FAQ
Username:	Register forum user name
Password:	Forgotten password?

(New message)

Subject:	game_loop(), select(), and dual core CPUs
Name:	Your forum user name. Register forum user name
Password:	Your forum password. Forgotten password?
Message:	Message to be posted (in English, please) Maximum of 6000 characters. Text only please, no HTML.
Forum codes:	Check this if your message uses 'forum codes' or templates (auto-detected for new posts). Forum codes Templates

Save this message ...

Subject review (reverse sequence)

Pages: 1 2 3 4 5

Posted by

Samson USA (683 posts)

Date

Thu 22 Feb 2007 02:37 PM (UTC) [ quote ]

Amended on Thu 22 Feb 2007 02:38 PM (UTC) by Samson

Message

I stick mostly to the philosophy of "if it works, it doesn't need fixing". Especially in cases where messing with it without really understanding what's happening under the hood can foul you up bigtime.

The descriptor handling in most Diku code seems to work just fine for the purpose. It takes input, performs updates, then sends output. The chain seems to flow pretty well since input should affect the outcome of update hanlders and those should in turn affect what gets output.

As long as the clock is reliable it should never be a problem. But as we saw, symptoms in your game are not always caused by your code :)

SmaugMuds.org: http://www.smaugmuds.org - The Smaug MUDs Community Center

"The past was erased, the erasure was forgotten, the lie became truth." -- George Orwell, 1984

Posted by

David Haley USA (3,881 posts)

bio Moderator

Date

Thu 22 Feb 2007 04:14 AM (UTC) [ quote ]

Message

Aww, I still think the main loop should be rewritten anyhow. :-) It would be pretty nice to have a proper separation of in-game actions and out-of-character commands. Still, that's a pretty big "should"...

David Haley aka Ksilyan
Head Programmer,
Legends of the Darkstone

http://david.the-haleys.org

Posted by

Nick Gammon Australia (18,770 posts)

bio Forum Administrator

Date

Thu 22 Feb 2007 04:10 AM (UTC) [ quote ]

Message

Or, needing to rewrite the main game loop <wink>.

- Nick Gammon

www.gammon.com.au, www.mushclient.com

Posted by

David Haley USA (3,881 posts)

bio Moderator

Date

Wed 21 Feb 2007 09:22 PM (UTC) [ quote ]

Message

Hmm. I'm booting using noapic and am not having timer issues, however I don't have acpi=of and no_timer-check.

Well, in any case, I'm glad that the problem was resolved in a very sane manner without 64 bits causing the world to end as we know it. :-)

David Haley aka Ksilyan
Head Programmer,
Legends of the Darkstone

http://david.the-haleys.org

Posted by

Samson USA (683 posts)

Date

Wed 21 Feb 2007 07:49 PM (UTC) [ quote ]

Message

I had:
ro root=LABEL=/ noapic acpi=off no_timer_check

Changed it to:
ro root=LABEL=/ acpi_use_timer_override

So far that's eliminated the issue.

SmaugMuds.org: http://www.smaugmuds.org - The Smaug MUDs Community Center

"The past was erased, the erasure was forgotten, the lie became truth." -- George Orwell, 1984

Posted by

David Haley USA (3,881 posts)

bio Moderator

Date

Tue 20 Feb 2007 05:33 PM (UTC) [ quote ]

Message

Which parameters were they out of curiosity? Was it noapic by any chance?

David Haley aka Ksilyan
Head Programmer,
Legends of the Darkstone

http://david.the-haleys.org

Posted by

Samson USA (683 posts)

Date

Tue 20 Feb 2007 01:41 PM (UTC) [ quote ]

Message

I say placebo because I can't imagine that we have THAT much data flowing around in every last possible situation. I could buy it from the overland display for immortals, but really, not much else we have should come anywhere close to spitting out 4K chunks.

Anyway, I figured people might like to know that it looks like the original problem has been fixed. Turned out that the clock timer issue was OS related, and it was due to boot arguments to the kernel that got left behind during the installation of the OS. It's been 4 days now and no sign of trouble from the clock on either server.

It was one of those half-fluke scenarios where the installer wouldn't run without special arguments, but the OS itself needed them removed after the installer was done. I'll definitely have to keep it in mind for the future.

SmaugMuds.org: http://www.smaugmuds.org - The Smaug MUDs Community Center

"The past was erased, the erasure was forgotten, the lie became truth." -- George Orwell, 1984

Posted by

David Haley USA (3,881 posts)

bio Moderator

Date

Tue 20 Feb 2007 07:01 AM (UTC) [ quote ]

Message

Quote:
But you are still ignoring the scenario of a user issuing "help <some massive help file>" 20 times or more per second. Instead you are actually giving priority servicing to users spamming the game with "out of game" commands. I would suggest that it may be a fundamental mud design flaw NOT regulating the user's rate of issuing commands, leaving yourself open to what amounts to in game DoS and spam attacks.

Trivial fix -- check the rate of output to prevent DoS attacks, and leave the speed-up for everybody else. Not a problem.

Quote:
Honestly I think throughout all this the number one improvement that I made was in sending 4K buffer chunks if the data exceeds 4K. For whatever reason, quite possibly even the placebo affect, things feel faster to me and anyone else who happens by.

It's not a placebo effect, it is in fact a very tangible speed-up. I'm not sure what else to say since I feel this horse has been beaten into undeadness and back a few times now. :)

David Haley aka Ksilyan
Head Programmer,
Legends of the Darkstone

http://david.the-haleys.org

Posted by

Samson USA (683 posts)

Date

Sat 17 Feb 2007 08:16 PM (UTC) [ quote ]

Message

Honestly I think throughout all this the number one improvement that I made was in sending 4K buffer chunks if the data exceeds 4K. For whatever reason, quite possibly even the placebo affect, things feel faster to me and anyone else who happens by.

SmaugMuds.org: http://www.smaugmuds.org - The Smaug MUDs Community Center

"The past was erased, the erasure was forgotten, the lie became truth." -- George Orwell, 1984

Posted by

Nick Gammon Australia (18,770 posts)

bio Forum Administrator

Date

Sat 17 Feb 2007 06:08 AM (UTC) [ quote ]

Message

I have to agree with that, and this is why, with faster CPUs, the existing design, which has theoretical problems, is probably totally acceptable in the current environment.

You may well find that you expend hundreds of hours "improving" it, with no noticeable performance improvements with existing player base numbers.

- Nick Gammon

www.gammon.com.au, www.mushclient.com

Posted by

Jon Lambert USA (26 posts)

Date

Sat 17 Feb 2007 05:01 AM (UTC) [ quote ]

Amended on Sat 17 Feb 2007 05:06 AM (UTC) by Jon Lambert

Message

> I think you would find that at a certain point (and I can't say what that is, maybe even 1000+ players),
> that the sleep/poll technique would become noticeably ineffecient.

Sure but I would suggest that someone planning, designing and writing a text mud server to support 1000+ players is extraordinarily optimistic, possibly even delusional. ;-)

Posted by

Jon Lambert USA (26 posts)

Date

Sat 17 Feb 2007 04:57 AM (UTC) [ quote ]

Message

> My original suggestion was that maybe the select with no descriptors might have been timing out
> prematurely, due to possible optimization, which is what the behaviour is under windows. Thus, that
> was relevant.

Fair enough as he did not specify which OS he was running. Windows select is not Posix compliant. If some version of Linux/BSD broke select() in that way I'm certain it would have generated thousands of bug reports within hours because it would have broken the large body of applications that depend on using select() in the same way Diku uses it, like say apache, rsync, subversion, mailman, kmail, openssl, etc.

> I think you will find however that any large online program (eg. a web server, MMORPG) would never
> build sleeps into the main loop, simply because they are going for optimal performance.

I'm not sure what you mean by large program. If you are talking about web servers and GRAPHICAL MMORPGs we are talking apples and oranges in terms of network architecture decisions.

Apache's main server process does select() sleep in between the select wait for accepting connections and waiting for child processes to end. However that's not even relevant to mud discusion because Apache's optimal architecture is in forking processes for each user (at least on Linux). That architecture would not work very well for muds at all because we share data among users.

On graphical MMOPRGS there are still different architectural considerations. In UOL for example almost all the world data resides on the client, and communications between the server and clients consists primarily of UDP broadcasting of very small packets, mostly under 1K chunks or so I was told. I understand that movement and collisions are resolved simultaneusly on both the client and server and that data syncronization and verification information is exchanged. These games also have front end load sharing code. Frankly the only thing I have to go on is past discussions I've had with some of the authors on mud-dev, as the source to such games like Ultima Online are not generally available for examination.

> The main justification for the "sleep select" seemed to be that it reduced the number of times select
> needed to be called, thus potentially reducing the times through the main loop.

Correct but consider the effects of not regulating user command rate. Other muds like MOO, Cool and ColdC move the command quota regulation out of the networking loop and into the system parse events, thus you see a single poll or select in the networking code. While I might think ColdC's architecture for example is in almost every way superior to a Diku it still doesn't make Diku's fundamentally broken.

Posted by

Jon Lambert USA (26 posts)

Date

Sat 17 Feb 2007 04:53 AM (UTC) [ quote ]

Message

>> So you've gained nothing from processing I/O faster than the commands are processed, except searching
>> the descriptor list dozens of times more every pulse. Oh wait, no that's not it, you've actually
>> optimized the handling of commands without a wait state. So the player typing "HELP STORY" 20 times a
>> second will have optimal servicing. Brilliant. ;-)

> I believe you misunderstood. I meant that players cannot get an advantage over other players in e.g.
> combat by typing in lots of commands in the hope to get multiple ones in per tick.

But you are still ignoring the scenario of a user issuing "help <some massive help file>" 20 times or more per second. Instead you are actually giving priority servicing to users spamming the game with "out of game" commands. I would suggest that it may be a fundamental mud design flaw NOT regulating the user's rate of issuing commands, leaving yourself open to what amounts to in game DoS and spam attacks.

> As for the output servicing, I think we've established over and over again that you don't give a hoot
> either way but it really bothers me to get choppy output.

No we've instead established that Derek Snider decided to chop the output of commands that produce large output. I've postulated that since he was running a mud with a fairly large amount of players, and quite possibly had either limited bandwidth or machine, he did it for performance reasons. I don't really think he needs to provide any proof that whatever tuning he did for his mud must necessarily apply to anyone else.

Secondly there is no need to personalize my defense of the Diku network architecture. I didn't write it and I don't run one.

> Let's forget about input for now. As I said, I can see potential problems in properly handling the
> input case. Let's focus on output. What I'd like to hear from you is simply why we should sit idle
> when there is output we could be sending.

No because you can't separate output from the input/update/output loop and discuss it in isolation because all of the output that will be generated comes from that input/update sequence. It doesn't get magically produced elsewhere. The only leftover output once that pulse loop ends is 1) that which would write block anyway, and 2) the output generated that exceeds the buffer size of the write routines. In the first case, that's what select was designed to test, there is really nothing to do but wait on those sockets to ripen. In the second case as already mentioned you simply tweak your write buffer. Now you aren't going to gain anything in implementing application write buffer sizes greater than the underlying OS sockets stack, which I'm guessing are still 32K for both input and output on most Unices, as you'll be hitting write block conditions again.

Now if you did an analysis and measurement, you are going to discover that 99% of the output generated from a stock Diku/Merc/Smaug Mud per pulse falls under the 4K buffer size. That's about a screen and 1/4 on your traditional 80x40 dummy console, or 5 screens per second. Let's not ignore that DikuMuds and kin are already delivering sub-second response time to their users.

Lastly in the realm of user response time, IBM did studies on response time on multi-user systems and discovered that response times greater than 2 seconds caused the attention of the user to drift and become less productive. That's not suprising and I would posit that "productive" might translate into distracted or annoyed when it comes to text gaming. But they also discovered that installations that provided sub-second but irregular response times to users received more user complaints about "lag" than when response times were uniform but longer.

Posted by

David Haley USA (3,881 posts)

bio Moderator

Date

Fri 16 Feb 2007 04:58 AM (UTC) [ quote ]

Message

Quote:
For more lightly loaded MUDs the existing technique may well be perfectly satisfactory.

For input, yes; but the delay in output can be quite noticeable. Maybe it doesn't bother some people (clearly Jon thinks it's fine) but to me it's very unaesthetic to have output come in chunky bits.

David Haley aka Ksilyan
Head Programmer,
Legends of the Darkstone

http://david.the-haleys.org

Posted by

Nick Gammon Australia (18,770 posts)

bio Forum Administrator

Date

Fri 16 Feb 2007 04:44 AM (UTC) [ quote ]

Message

Effectively the dual-select technique is a variant of doing IO by polling. That is, you sleep for a while (to give the CPU a chance to do something else), and then you poll the IO list to see if there is anything to do.

I accept that this works on lightly to medium loaded MUDs, as obviously everyone is happy with it.

I think you would find that at a certain point (and I can't say what that is, maybe even 1000+ players), that the sleep/poll technique would become noticeably ineffecient.

For more lightly loaded MUDs the existing technique may well be perfectly satisfactory.

- Nick Gammon

www.gammon.com.au, www.mushclient.com

The dates and times for posts above are shown in Universal Co-ordinated Time (UTC).

To show them in your local time you can join the forum, and then set the 'time correction' field in your profile to the number of hours difference between your location and UTC time.

13,160 views.

This is page 1, subject is 5 pages long: 1 2 3 4 5

Reply to this subject Start a new subject Refresh page

Go to topic: Search the forum

Comments to: Gammon Software support
Forum RSS feed ( http://www.gammon.com.au/rss/forum.xml )