Register forum user name Search FAQ

Gammon Forum

Notice: Any messages purporting to come from this site telling you that your password has expired, or that you need to verify your details, confirm your email, resolve issues, making threats, or asking for money, are spam. We do not email users with any such messages. If you have lost your password you can obtain a new one by using the password reset link.

Due to spam on this forum, all posts now need moderator approval.

 Entire forum ➜ SMAUG ➜ SMAUG coding ➜ Who knows windows threads (or any threads)
Who knows windows threads (or any threads)

It is now over 60 days since the last post. This thread is closed.     Refresh page

Pages: 1 2  

Posted by ThomasWatts   USA  (66 posts)  Bio
Date Sat 10 Nov 2007 01:59 PM (UTC)
Message
Separated the networking and the lua into two threads. One for the networking, and one for the lua world. Both do things and then sleep, the networking runs at 4 pulses a second, the lua 2. I can connect, it negotiates telnet options, I can sometimes type in a username/password and sometimes more, but eventually I will be booted with an error on the lua side, or even a crash with on the c side. Is there a way to (A) protect the lua stack while still allowing other functions to run, or (B) a decent time-sharing strategy?

In short, anyone worked with threads before and had similar to the above problems? Or should I just stick with a single thread?
Top
Posted by David Haley   USA  (3,881 posts)  Bio
Date Reply #1 on Sat 10 Nov 2007 09:18 PM (UTC)
Message
Sure: you need two locks, one for the Lua state and one for the networking stuff. Whenever one thread tries to do something to the Lua state, you should acquire the Lua lock. Whenever one thread tries to do something to the networking state, you acquire the networking stuff lock. (And of course, when you're done with the tick's processing, you should release the lock.)

This way, only one thread can do something to the state at once, which will prevent the kind of corruption you're talking about. You can model this differently, e.g. have more fine-grained synchronization to make it more efficient, but this is the simplest model and is probably sufficient.
David Haley aka Ksilyan
Head Programmer,
Legends of the Darkstone

http://david.the-haleys.org
Top
Posted by Nick Gammon   Australia  (23,133 posts)  Bio   Forum Administrator
Date Reply #2 on Sat 10 Nov 2007 09:59 PM (UTC)
Message
However this sort of thing is hard to get right. You have potential race conditions and deadly embraces.

For example, if the Lua lock is taken, and then calls the network code (eg. to send a message) and some error condition arises that needs to execute Lua code, then the Lua lock is already taken and the whole process halts.

Personally I would stick to a single thread, the complexities of maintaining multiple locks is probably not worth it.
- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top
Posted by David Haley   USA  (3,881 posts)  Bio
Date Reply #3 on Sat 10 Nov 2007 10:08 PM (UTC)

Amended on Sat 10 Nov 2007 10:45 PM (UTC) by Nick Gammon

Message
For this reason I would use more of a producer-consumer type model. The Lua thread should build up a list of things that need to be sent over the network. Then, the network thread should grab that stuff on its tick and send it -- and therefore, that processing is not occurring during the Lua thread execution. So, if you need to handle errors in Lua, you don't run into the deadlock that Nick described. (And similarly for the network thread producing stuff for the Lua thread to later consume.)

I agree though that this adds a lot of complexities and it's worth thinking why exactly you want to break it into threads. In general, you need threads in the following kind of situation:

- one operation needs to run and it takes a long time to run
- the system needs to remain responsive during that time

Classic example is a GUI application with some processing going on where it needs to remain responsive to e.g. a cancel button being pressed.

In MUD terms, what operations are you running that take a long time? I can only think of one, really: the reverse IP lookup, which sometimes stalls the game for many seconds at a time.

And in a MUD, the system doesn't have to be actually responsive, it just has to give the impression of being responsive. That is, you don't need to handle input as soon as it arrives, you just need to do it frequently enough that it looks like you are. (This is why SMAUG can get away with just a single thread.)

So yes, I would think carefully about why you want threads. I personally think that threads make for a more natural model of the program in some cases (you might note that your proposed model is very similar to the engine spec thoughts I wrote on the BabbleMUD page), and for me that is a point in their favor, but the immediate point against is that it adds the complexity of synchronization. So you need to think carefully about how the threads will communicate with each other in order to avoid cases like Nick described.
David Haley aka Ksilyan
Head Programmer,
Legends of the Darkstone

http://david.the-haleys.org
Top
Posted by Nick Gammon   Australia  (23,133 posts)  Bio   Forum Administrator
Date Reply #4 on Sat 10 Nov 2007 10:48 PM (UTC)
Message
Quote:

And in a MUD, the system doesn't have to be actually responsive, it just has to give the impression of being responsive.


And in fact you could make a case for throttling the server, in the sense that, if you carefully write it to be really really responsive, then it could handle 1000 commands per second spewed out by a client bot, to the disadvantage of "real" players.

So, if you are going to build in, say, that it only allows 2 commands per player per second, you are hardly going to need (with today's fast PCs) to go to heaps of trouble to get the extra microsecond's performance out of your CPU.
- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top
Posted by David Haley   USA  (3,881 posts)  Bio
Date Reply #5 on Sun 11 Nov 2007 12:23 AM (UTC)
Message
That's a very good point too. What it comes down to is that you shouldn't really be thinking of using threads for performance, [i]except[/i] for operations that take a very long time during which you must continue to be responsive. There are just two operations in SMAUG that I know of that do this:

(1) the IP lookup I mentioned
(2) the object finding spell that was previously badly implemented so that it was very slow

Basically threads shouldn't be added just for the sake of having threads. There should be a very good reason to have them, the best of which I can think of is to avoid the system locking up during long & slow operations. (Where "long" is long enough for the game to stop being "normally responsive" to players. A second or two is probably enough to be considered "long".)
David Haley aka Ksilyan
Head Programmer,
Legends of the Darkstone

http://david.the-haleys.org
Top
Posted by Samson   USA  (683 posts)  Bio
Date Reply #6 on Sun 11 Nov 2007 05:49 AM (UTC)
Message
And as of FUSS 1.8, the DNS lookup issue is no longer a problem either, due to the use of a forked process that handles the lookup without stalling the game.
Top
Posted by ThomasWatts   USA  (66 posts)  Bio
Date Reply #7 on Sun 11 Nov 2007 08:47 AM (UTC)
Message
Yeah, this was just a case of lets see what I can do with this of thing, over any real need. I doubt anyone will ever play on what I'm designing, I just enjoy futzing with stuff. The comments are great and give me ideas, I may try a message system between the two using simple single linked lists. Each thread has a stack that is added to that is passed to the other thread. The thread doing the reading only passes back where it left off. At least that's idea to cut down on race conditions. I'll let everyone know what happens.
Top
Posted by David Haley   USA  (3,881 posts)  Bio
Date Reply #8 on Sun 11 Nov 2007 09:06 AM (UTC)
Message
I look forward to hearing about the results of your attempts. :) Like I said, the two-thread approach seems very natural to me, even though it adds technical difficulty in some places, and even though I'm not sure what exactly it would gain re: performance etc. :-) I've been meaning to try it for a while now, but have just never gotten around to it.
David Haley aka Ksilyan
Head Programmer,
Legends of the Darkstone

http://david.the-haleys.org
Top
Posted by ThomasWatts   USA  (66 posts)  Bio
Date Reply #9 on Sun 11 Nov 2007 11:37 AM (UTC)
Message
As of this moment, after try number three, it seems too much hassle.
The first try was documented above.
The second try was the message system mentioned in my previous post. The race conditions just moved from lua stack corruption to linked pointer corruption (although it took a lot of alt-tabbing and keyboard mashing for this to happen). I perhaps could have done message passing different, but that's just what I tried.
The third try as of 30 minutes ago was a rather simplistic locking mechanism. The best of the three tries to be sure, but it didn't seem to always work 100%. As with try #2, if I knew more about threads and race conditions, perhaps this could have worked.
As it stands I'm just going to see how a single thread goes for now. Input 4 times a second, and the world tick 2 times everything shouldn't get bogged down unless I really write some terrible lua. We shall see in the future.
If anyone else tries for multi-threaded, I'm sure all of us will want to know how it goes.
Top
Posted by David Haley   USA  (3,881 posts)  Bio
Date Reply #10 on Sun 11 Nov 2007 10:34 PM (UTC)
Message
Just to double-check, what kind of synchronization were you using with these?
David Haley aka Ksilyan
Head Programmer,
Legends of the Darkstone

http://david.the-haleys.org
Top
Posted by Isthiriel   (113 posts)  Bio
Date Reply #11 on Sun 11 Nov 2007 11:03 PM (UTC)

Amended on Sun 11 Nov 2007 11:06 PM (UTC) by Isthiriel

Message
Quote:
ThomasWatts wrote:
Separated the networking and the lua into two threads. One for the networking, and one for the lua world. Both do things and then sleep, the networking runs at 4 pulses a second, the lua 2.

I don't know why you would want pulses for the networking or the lua.

The big advantage of a thread-driven model is you don't have to have a polling loop. The network code can sleep (select()) until it has input to process and the lua game state should be mostly event driven?

Quote:
Nick Gammon wrote:
For example, if the Lua lock is taken, and then calls the network code (eg. to send a message) and some error condition arises that needs to execute Lua code, then the Lua lock is already taken and the whole process halts.

That's what reentrant locks are for.

Though if you are doing threading the lua code should dispatch an event to the network code, release its locks and sleep until the response is ready? (Or otherwise keep itself occupied doing other stuff rather than blocking until the network code is finished.)

Quote:
David Haley wrote:
I agree though that this adds a lot of complexities and it's worth thinking why exactly you want to break it into threads. In general, you need threads in the following kind of situation:

Or if you're mud is processor intensive (lots of scripting code, lots of players, mccp, physics?) AND it running on a multi-core (or multi-processor) box.

Ideally, you want 1 thread/processor core.

Quote:
David Haley wrote:
Classic example is a GUI application with some processing going on where it needs to remain responsive to e.g. a cancel button being pressed.

The definition of "responsive" varies too, a GUI should respond in less than 1/10 of a second, but a MUD that might have a ping of up to 300ms can be called "responsive" if it takes 2-3 seconds between entering a command and seeing it enacted.

Quote:
David Haley wrote:
In MUD terms, what operations are you running that take a long time? I can only think of one, really: the reverse IP lookup, which sometimes stalls the game for many seconds at a time.

Can't that be done asynchronously? Most of the time taken is blocking waiting for a response from the DNS server?

Quote:
Nick Gammon wrote:
And in fact you could make a case for throttling the server, in the sense that, if you carefully write it to be really really responsive, then it could handle 1000 commands per second spewed out by a client bot, to the disadvantage of "real" players.

Every command should take some "time" to enact, you might be able to handle 1000 commands/sec/player (and it's something to aim for, since 1000 commands/sec on a playerbase of 1 is only 5 commands/sec on a playerbase of 200) but you should only allow a queue of 20 or so commands and each command should take at least 1/16 of a second (for preference changes or other OOG things that translate into a single function call) up to 4 or 5 seconds (for time consuming IG activities... searching, digging, expensive spells, repairing, crafting). Longer than 5 seconds where the game seems unresponsive (because your character is busy) can be problematic :(

Quote:
ThomasWatts wrote:
As it stands I'm just going to see how a single thread goes for now. Input 4 times a second, and the world tick 2 times everything shouldn't get bogged down unless I really write some terrible lua. We shall see in the future.
If anyone else tries for multi-threaded, I'm sure all of us will want to know how it goes.

Python has an implementation called "Stackless", I don't know if Lua has something similar?

Stackless is single threaded BUT provides lots of tools for cooperative multitasking with "microthreads" or "green threads". The idea is that instead of being preempted by the OS scheduler, individual tasklets hand off to each other so you don't need to worry about locks because there is only one thread, and you get all the advantages (except multi-processor use) provided each tasklet hands off the execution when it no longer needs it (or periodically even if it does).
Top
Posted by ThomasWatts   USA  (66 posts)  Bio
Date Reply #12 on Sun 11 Nov 2007 11:13 PM (UTC)
Message
That's why I was asking about who knew what about threads and thread control. I know very little about the subject beyond simple thread creation and non-interacting threads. Mainly from when I was taking college classes few too many years ago.
I was using pathetic boolean checks for locks and a message queue for when the locks were engaged. I tried either/or and a system of both, and neither solved the eventual problem of race conditions. It was alot of hassle that I decided to not deal with at the time.
Also about being stackless, lua is not. Each C function has a stack available of at least 20 in size. However, when you are using the same lua state, you use the same lua stack. So when the inevitable happens the C statements end up altering the same stack. So bad st*ff happens.
Top
Posted by David Haley   USA  (3,881 posts)  Bio
Date Reply #13 on Sun 11 Nov 2007 11:46 PM (UTC)
Message
Quote:
Isthiriel:
That's what reentrant locks are for.

I believe he was describing a situation where the Lua thread needs to lock on the networking thread, and then the networking thread needs to lock on the Lua thread. Poof, deadlock. So no, it can't be solved by reentrant locks. It's a classic deadlock problem, like dining philosophers.

Quote:
Isthiriel:
Or if you're mud is processor intensive (lots of scripting code, lots of players, mccp, physics?) AND it running on a multi-core (or multi-processor) box.

Ideally, you want 1 thread/processor core.

Well, only if it actually matters. You don't need threads unless you're doing long, blocking operations during which the rest of the program needs to keep on going. Just because you have a bunch of cores doesn't mean you should spawn one thread for each of them. (In other words, there's no reason to be multi-threaded just because you have several processors.)

Quote:
Isthiriel:
Can't that be done asynchronously? Most of the time taken is blocking waiting for a response from the DNS server?

Yes, in fact that's what I believe FUSS 1.8 does. Still, you would need some kind of synchronization, depending on what exactly is being done.

Quote:
Isthiriel:
Python has an implementation called "Stackless", I don't know if Lua has something similar?

Yes, they call them coroutines. But they're not "stackless"; each coroutine still has its own stack. I suspect that the term "stackless" is something of a misnomer, because each of these "tasklets" (in Python terminology) would have their own stacks, no?

Quote:
ThomasWatts:
I was using pathetic boolean checks for locks and a message queue for when the locks were engaged.

Well that's not going to work. :-P You need to use the actual synchronization primitives like semaphores and locks (aka mutexes). Whatever threading library you're using will provide something like that.
David Haley aka Ksilyan
Head Programmer,
Legends of the Darkstone

http://david.the-haleys.org
Top
Posted by Nick Gammon   Australia  (23,133 posts)  Bio   Forum Administrator
Date Reply #14 on Mon 12 Nov 2007 04:03 AM (UTC)
Message
Quote:

Every command should take some "time" to enact, you might be able to handle 1000 commands/sec/player (and it's something to aim for, since 1000 commands/sec on a playerbase of 1 is only 5 commands/sec on a playerbase of 200) ...


I think there is a slight misunderstanding here. The way you have written that, you are saying the server should handle 1000/commands/sec overall (not per player) to give 5 commands/sec for a single player, if the playerbase is 200.

I don't think you need to design in to handle 1000 commands/sec/player as that seems to me to be abuse of the server. How many genuine players are going to enter 1000 commands in one second?

Quote:

Python has an implementation called "Stackless", I don't know if Lua has something similar?


It is fundamental to Lua that each script coroutine has its own "Lua" stack, I am not sure if this is exactly what you mean.

I believe Lua is supposed to be potentially thread safe, although there may be compile-time considerations. For example, the default I believe is to use malloc and free, and I am not sure if they are thread reentrant.

In any case, depending on your design, if you preempted a Lua script execution, and then did something else with the script state, I would personally expect it to crash. A script is supposed to run until it releases control. Obviously process-level preemption would occur all the time, but at a thread level I would be cautious, if the preempting thread does anything at all with the script state - or indeed, if it alters the execution of a C routine that the script has called.

Here is an example of what I am talking about. Say we have a Lua script like this:


if mob_exists (12345) then
  move_mob (12345, 567)  -- move to different room
end -- if


Now imagine that a thread preemption occurs after the 'if' is successfully tested, but which then deletes the mob. That way the results of the if are incorrect, unknown to the script.

- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top

The dates and times for posts above are shown in Universal Co-ordinated Time (UTC).

To show them in your local time you can join the forum, and then set the 'time correction' field in your profile to the number of hours difference between your location and UTC time.

59,572 views.

This is page 1, subject is 2 pages long: 1 2  [Next page]

It is now over 60 days since the last post. This thread is closed.     Refresh page

Go to topic:           Search the forum

[Go to top] top

Information and images on this site are licensed under the Creative Commons Attribution 3.0 Australia License unless stated otherwise.

Home