[Home] [Downloads] [Search] [Help/forum]

Gammon Software Solutions forum

See www.mushclient.com/spam for dealing with forum spam. Please read the MUSHclient FAQ!

[Folder]  Entire forum
-> [Folder]  Programming
. -> [Folder]  General
. . -> [Subject]  Combining Ruby and C++
Home  |  Users  |  Search  |  FAQ
Username:
Register forum user name
Password:
Forgotten password?
(New message)
Subject: Combining Ruby and C++
Name:
Your forum user name.
Register forum user name
Password:
Your forum password.
Forgotten password?
Message:
Message to be posted (in English, please).
Forum codes:
Check this if your message uses 'forum codes' or templates (auto-detected for new posts).
Forum codes Templates

Save this message ...

Subject review (reverse sequence)

Pages: 1 2  

Posted by Nick Gammon   Australia  (18,770 posts)  [Biography] bio   Forum Administrator
Date Fri 04 Apr 2008 09:34 PM (UTC)  [ quote  ]
Message
This post shows you to embed Lua in your own application:

http://www.gammon.com.au/forum/?id=6400

It shows how in a few lines of code you can be executing a Lua script. And in a few more you can read in a Lua script file and execute it.

Meanwhile this post shows how to add extra functions to the Lua script space (that is, functions written in C):

http://www.gammon.com.au/forum/?id=4915
- Nick Gammon

www.gammon.com.au, www.mushclient.com
[Go to top] top
Posted by David Haley   USA  (3,881 posts)  [Biography] bio   Moderator
Date Fri 04 Apr 2008 09:01 PM (UTC)  [ quote  ]
Message
You know, I'm in the process of learning Ruby, and it seems that many of the concepts are very similar. Ruby functions that take blocks, like the 'each' function, are rather like coroutine iterators (the Lua manual explains these). Ruby duck-typing can be achieved by testing if an object field is nil. (This works except in extreme cases of meta-tabling.) Lua doesn't have the direct support for classes that Ruby has, but using metatables you can easily accomplish much of the same functionality: with a little work, you can use metatables to completely replicate a full class system with inheritance etc.

The point of this being that if you know and understand Ruby, it should be fairly easy to adapt to Lua. At least, that's what I'm finding as I go in the other direction. :-)

The book Programming in Lua is excellent for getting an introduction to Lua:
http://www.lua.org/pil/

It's a superb book, I highly recommend it. And it's available online for free (although I have hard-copies of both versions; they're nice to have and I like supporting the development effort even in a small way).
David Haley aka Ksilyan
Head Programmer,
Legends of the Darkstone

http://david.the-haleys.org
[Go to top] top
Posted by Nick Cash   USA  (626 posts)  [Biography] bio
Date Fri 04 Apr 2008 08:55 PM (UTC)  [ quote  ]
Message
It is plainly known to ruby programmers (and is written right in the pickaxe) that ruby was not originally designed with embedding in mind, so its embedding interface is seriously hackish and unhelpful. Its extending interface is much better it seeems.

I will look into switching to Lua. There is no point wasting my time trying to embed Ruby when it could be done far easier in another language. Although I like Ruby, it seems it won't suit this purpose as much as I thought it would.

Thanks for the feed back. :)
~Nick Cash
http://www.nick-cash.com
[Go to top] top
Posted by Nick Gammon   Australia  (18,770 posts)  [Biography] bio   Forum Administrator
Date Fri 04 Apr 2008 08:31 PM (UTC)  [ quote  ]
Message
Quote:

Calling ruby one way never gives control back to the C/C++ program. Calling it the other way (in which you call individual ruby functions it seems) would work ok, however, it seems it wouldn't remember the stack pointer or anything else in the script space between calls.


Honestly, if this is true, I would ditch Ruby.

Lua at least is designed to be embedded. A whole chapter in the "Learning Lua" book and most of the Reference Manual describes how to get data in/out of Lua.

Once you initially create the script space, you can call Lua functions in it as often as you want, and each time around it remembers how you "left things" from last time. For example, if you make a variable, it stays made.

You can even leave mid-function by using coroutines. By doing a coroutine.yield you are telling Lua that you have reached a logical point to return (to C++ or simply your calling function in Lua). Subsequent calls into that script space can resume that function, when appropriate.

This is how you can and would implement things like doing a login dialog ("what is your name?", "what is your password?" and so on).
- Nick Gammon

www.gammon.com.au, www.mushclient.com
[Go to top] top
Posted by David Haley   USA  (3,881 posts)  [Biography] bio   Moderator
Date Fri 04 Apr 2008 05:35 PM (UTC)  [ quote  ]
Message
Quote:
However, based on my reading, there is a big problem. Calling ruby one way never gives control back to the C/C++ program.

Really? I'd be pretty surprised if it were impossible to pass control back and forth. In Lua and Perl -- the two languages I have experience embedding -- as soon as the function you called returns, control is restored to the C/C++ caller.

Calling the parser might leave control in the scripting language "forever" because it will start parsing standard input, and will only finish when standard input is closed.

Now, I don't have any experience embedding Ruby, but I would be surprised (and disappointed) if you had to use threading and synchronization to get this working.
David Haley aka Ksilyan
Head Programmer,
Legends of the Darkstone

http://david.the-haleys.org
[Go to top] top
Posted by Nick Cash   USA  (626 posts)  [Biography] bio
Date Fri 04 Apr 2008 04:06 AM (UTC)  [ quote  ]

Amended on Fri 04 Apr 2008 05:08 AM (UTC) by Nick Cash

Message
I have read a great deal and I fully understand the difference spaces. I think perhaps I need to work on some small example programs to truly understand how it is all going to be put together.

However, based on my reading, there is a big problem. Calling ruby one way never gives control back to the C/C++ program. Calling it the other way (in which you call individual ruby functions it seems) would work ok, however, it seems it wouldn't remember the stack pointer or anything else in the script space between calls.

So, after viewing these methods, I think I'm stuck with threading. I've seen an example where the fellow setup a ruby thread, synchronized it, and periodically had it sleep so the main program could do things. It still seems like there should be a better way.

I guess what I'm envisioning is loading the C++ program, initializing the server and what not, start ruby and initial the rest of the mud (areas, etc), then periodically toss control from ruby over to the core C++ server to process sockets, then head back to ruby to handle the rest of the game. Initially I was thinking I would just attach scripts to certain objects on the c++ side so they could run after certain events, but giving the embeding options I would need to a lot of work to make it workable for NPCs or other in-game entities.

I don't like the idea of scraping what I have because it is pretty good code, but it seems like making a ruby server and merely extending it to handle sockets might be the way to go since database connections can be done easily in ruby as well.
~Nick Cash
http://www.nick-cash.com
[Go to top] top
Posted by Nick Gammon   Australia  (18,770 posts)  [Biography] bio   Forum Administrator
Date Thu 03 Apr 2008 08:57 PM (UTC)  [ quote  ]

Amended on Thu 03 Apr 2008 08:58 PM (UTC) by Nick Gammon

Message
I presume he means a script that the engine calls, written in Ruby. Thus the game engine fires up, loads and compiles the script, accepts incoming connections, and then looks for appropriate functions in the script file to handle them (eg. function "HandleNewConnection").

As for the variables, well this is the big issue. Once you have a script engine you have multiple variable "spaces". MUSHclient is a classic example of that, where there is the C++ variable space which holds things like the scrollback buffer (amongst many others), and the variable spaces for each script engine (eg. one per world, and one per plugin).

You need to provide some sort of interface to get/receive from one space to another. For example, in MUSHclient you can use GetInfo, to find out, in a script, things that are stored in the main C++ variables. Then you might use SetVariable / AddTrigger / AddAlias to let a script cause C++ variable(s) to be changed. You might make interfaces that work the other way, in MUSHclient plugin callbacks are effectively that, as the main C++ code calls a plugin script (eg. on world connect) giving it the opportunity to modify script variables.

The postings I did recently about adding Lua to SMAUG show how you might interface a script engine with a C engine. Again, I had to make interfaces that let each "side" talk to each other.

You might want to minimize the amount of information that needs to be exchanged between script side and C++ side. One way is to store almost everything on the script side, or alternatively store most things on the C++ side.

There are pros and cons for both approaches, some considerations would be:


  • Speed
  • Flexibility
  • Whether recompilation is needed if you change something
  • Ease of coding
  • Ease of maintenance
  • Ease of debugging
  • Portability
  • How errors are handled
  • Reliability
  • Memory use


In my SMAUG implementation I actually used a separate script space per player. As David and others have pointed out, this is rather inefficient of memory, however it has certain advantages. Ease of coding, ease of recovery from errors that only affect one player are a couple of them.

However this approach would break somewhat in situations (like combat) where you needed simultaneous access to variables for multiple players / NPCs.

[EDIT] My reply was being written as David posted his, so I don't address anything David said above.
- Nick Gammon

www.gammon.com.au, www.mushclient.com
[Go to top] top
Posted by David Haley   USA  (3,881 posts)  [Biography] bio   Moderator
Date Thu 03 Apr 2008 08:52 PM (UTC)  [ quote  ]
Message
The Quake2 system used a DLL because that is the easiest (only?) way in C to swap out the game logic while not letting modders touch the core engine. When you installed a mod, you would use a different DLL, which implemented the game callbacks as it saw fit.

For instance, to make bouncing rockets, you would change the source code for the rocket collision callback, and recompute the new velocity vector based on impact vector etc.

For a scripting language -- Ruby, Lua, whatever -- you probably wouldn't use the DLL approach because you can load a script when the program starts. So the script becomes the DLL. That script would (presumably) load up other scripts to implement the game logic as needed.

I think that players and NPCs would be treated the same way. For example, your core engine could have the notion of a "game entity" which could be an object, a room, a character (I prefer the term 'actor' personally), etc. Let's consider actors: an actor could have a callback for damage being inflicted on that actor. That callback would be implemented in the scripting language by a function called (e.g.) handle_damage that takes as arguments (e.g.) the actor inflicting damage, the victim of the damage, the amount of damage being dealt, the type of damage, etc.

For accessing data, Quake2 solved it by having a C structure the first part of which you could not touch. It used a clever trick of C whereby you can "add" fields to a structure by having another structure the first part of which is the original structure but that adds its own fields. Since the fields are laid out in the same order, anything that uses the new structure can be oblivious to the fact that there are new fields, because the fields it knows about will be at the same memory offsets from the structure beginning.

If that didn't make any sense, basically Quake2 sticks all data in the same C structure, and you have access to it from C.

Scripting languages make things complicated because you actually have two separate spaces in which to store data. One solution is to have it all in C++ and use accessors to grab the data; that is complicated because you need to translate the data appropriately etc. Another solution is to put everything in the scripting language, but then you need to access it from the C++ side, which also entails translation.

I think that in practice, however, the core data will not be accessed that often by the game logic. There is a relatively small number of functions that need to know about the core concepts, but a remarkable amount of logic can be implemented with relatively little access to actual core data. A combat system, for instance, doesn't need to know that much about the core data. (That depends, I suppose, on what data is considered "core"...)

And yes, I agree: this is a very hard design problem... I've been thinking about it for many years and have yet to find a solution that I truly like. I suspect though that this might be a case of "analysis paralysis" and the best thing might be to just forge ahead and see what happens, learning from mistakes as necessary.
David Haley aka Ksilyan
Head Programmer,
Legends of the Darkstone

http://david.the-haleys.org
[Go to top] top
Posted by Nick Cash   USA  (626 posts)  [Biography] bio
Date Thu 03 Apr 2008 08:32 PM (UTC)  [ quote  ]
Message
The Quake2 engine sounds similar to what I would like. The problem I'm having I guess is wrapping my head around how exactly this will all work. Are you suggesting the logic and everything is stuck into a DLL that the engine calls? This seems like a far more elegant solution, but I am unsure how to use Ruby to generate a DLL that will interface with C++.

This seems like it would work well in general for players, but knowing I will have NPCs and other things allocated, how would the Ruby code in the DLL get access to that data?

This may be the most complex design topic I've ever faced. So very interesting, yet confusing at times.
~Nick Cash
http://www.nick-cash.com
[Go to top] top
Posted by David Haley   USA  (3,881 posts)  [Biography] bio   Moderator
Date Thu 03 Apr 2008 04:46 AM (UTC)  [ quote  ]
Message
I agree that it's extremely important to make a decision about where the data lies, and then stick to it. Having data spread out over the two languages will be very messy. If the data is C++, you need to have a good API in Lua for manipulating it.

I never really did figure out a way to handle this that I like a lot. Storing everything in Lua is very good at the beginning but I find it hard to grow due to the lack of compile-time checking. But storing it in C++ -- which gets you compile-time checking -- means you lose part of the point of using Lua in the first place, which is the ability to rapidly add fields and other data at runtime.

I'm thinking that you could get some degree of runtime checking by using metatables, or a static analysis tool of some sort, but the former might be expensive in overhead whereas the latter would be difficult to implement.

A hybrid approach could be made to work if you use metatables cleverly, but as Nick said you'll have quite a bit of work to get data from one end to the other.

The Quake2 engine used the following rough separation:

1. Core engine (you couldn't touch this)
Contained fundamental physics of objects: positions, time callback pointers, collision callback pointers, velocities, etc.

2. Game logic (you had the C .dll source for this)
Implemented the above callbacks; implemented the command handlers for pressing keys and so forth.



Basically, the core engine provided the framework, and you added your game-specific stuff to the game logic DLL.

In a MUD setting, I suppose this would be comparable to laying down the physics of rooms, exits, world position, etc. in the core engine (presumably the C++ side of things) and then having everything else (HP, mana, etc.) be in Lua.
David Haley aka Ksilyan
Head Programmer,
Legends of the Darkstone

http://david.the-haleys.org
[Go to top] top
Posted by Nick Gammon   Australia  (18,770 posts)  [Biography] bio   Forum Administrator
Date Thu 03 Apr 2008 02:59 AM (UTC)  [ quote  ]
Message
Ked's earlier reply is still relevant. You need to decide if you want most of your code in Ruby (or Lua or whatever), and extend that will some C++ code, or have most of your code in C++ and call a few scripts here and there to do things like mob progs.

Lua has excellent interfacing methods (and they are well documented too), because it was really designed to be interfaced from the start. You can do what MUSHclient does, and have most code in C++, and call Lua scripts from time to time.

Or, you can do what I did with an experimental Lua MUD, and have virtually everything in Lua, with some support code in C++ where required, for example, sockets.

If you go the hybrid approach you have problems with getting stuff (eg. details about what a mob is currently doing) from the C++ side to the scripting side, and back again. That is why I felt more comfortable doing virtually everything in Lua.
- Nick Gammon

www.gammon.com.au, www.mushclient.com
[Go to top] top
Posted by Nick Cash   USA  (626 posts)  [Biography] bio
Date Thu 03 Apr 2008 01:42 AM (UTC)  [ quote  ]
Message
I've actually come to a good point on this (finally) to ask more questions. I have sockets, logging, and database support programmed in C++. I'm at the point where I truly need to embed the interpreter before I continue for things to be as dynamic and amazing as possible.

Since most of my data is stored in a MySQL database, I see little reason to interface ruby and C++ explicitly other then the need for ruby to print messages to the socket on occasion, which only requires ruby to pass a string to a single C++ function. However, knowing I will have instances of rooms, objects, npcs, and perhaps other things allocated in C++ (all presumably with their own logic scripts), it seems like the wrong approach to take. The ruby embedding interface is pretty awful, and it gives me the option of losing control over my program and turning it over to the interpreter, or having to hard code every script I'd like to call, not to mention none of the scripts will have any access to data allocated in C++.

Is there a better solution? Does lua implement better? Should I work on this the other way around, programming most everything in ruby and extending the ruby code so sockets and what else I have are in C/C++?

I don't really want to be dependent on foreign libraries, such as SWIG. If there isn't an elegant way to do it I'll just have to make one.
~Nick Cash
http://www.nick-cash.com
[Go to top] top
Posted by Nick Gammon   Australia  (18,770 posts)  [Biography] bio   Forum Administrator
Date Wed 09 Aug 2006 10:35 PM (UTC)  [ quote  ]
Message
Quote:

We want the core functionality of the game to be in C++, mainly socket handling. Almost all other game logic would be handled in Ruby scripts.


I was planning something similar, although I think "core functionality" depends on your definition. If "almost all" logic is in Ruby, I would say your core functionality is in Ruby.

I have been experimenting with a Lua MUD (I know this isn't Ruby, but the idea is similar). I obtained a DLL from the Lua downloads site that provides all the TCP/IP functionality I need. In fact, to load that, all I had to do is this:


  print "Loading sockets library..."
  
  -- load  luasocket namespace
  socket = require("socket")
  print ("Socket library version:", socket._VERSION)


From then on, I was able to poll for incoming connections, see if there was text arriving, or send outgoing text, all in Lua, using the socket library.

After that, for reasons I described earlier, all the data storage was in Lua tables, so the need to convert to/from C++ didn't arise.

I also used the coroutines functionality in Lua to make processing new players much easier. Rather than have the rather convoluted "nanny" routine that SMAUG uses, with all its states, and having to remember what the player typed last time through, it can be done by making the whole connection process into a coroutine, and then yielding when more input from the player is required. See below for how I did that:


function NewConnection (line, client)
local name

  Send (client, messages.welcome)
  
  -- get name
  while true do
    Send (client, "Your name? (Enter 'new' to create a new character) ...", true)
    name = coroutine.yield ()

    local ok, err = validname (name)
    
    if ok then
      name = capitalize (name)
      if name == "New" then
        break
      end -- special name
      
      if yes_no (client, "You are called " .. name .. ", right? ...") then
        break
      end -- they agreed to that name
      
    else
      Send (client, err)
      
    end -- good name entered
      
  end -- while getting name

  local char -- the character s/he is
  
  if name == "New" then
    char = NewCharacter (client)
  else
    char = ExistingCharacter (client, name)
  end   

  if not char then
    Send (client, "Press <enter> to restart the connection process ...")
    return
  end
    
  print (char.name, "has entered the game.")
  Send (client, "Welcome to " .. config.mudname)
  char.connected_time = os.time ()
  char.client = client  -- point our character to the controlling client
  client.char = char    -- tell the client which character it is
  char.playing = true   -- they are on now
  client.handler = CommandHandler

end -- NewConnection



Notice how I simply ask their name and yield until I get a response? The low-level TCP socket handler simply breaks input into lines, and when a line has arrived resumes the current coroutine with that line. This could be the "new player" handler, or simply the command processor.

We can do a similar thing to get a yes/no response from the player (as called from above):


-- get a yes/no reponse from the client
function yes_no (client, query)
local reply

  while true do
    Send (client, query, true)
    reply = trim (coroutine.yield ())
    
    if reply == "yes" or reply == "y" then
      return true
    elseif reply == "no" or reply == "n" then
      return false
    end -- if good reply
    Send (client, "Please reply 'yes' or 'no'.")
  end -- while loop
  
end -- yes_no


This sends the query, and then yields until it gets a reply. Then if the reply is not yes/y/no/n then it resends the query and loops. Nice and easy to see how it is working.

I'm not sure if Ruby has coroutines, and if not maybe consider Lua instead. A quick search on the web seems to indicate that Ruby does have coroutines, and probably an interface for TCP/IP as well, so that is probably all you really need.

Quote:

I am unsure exactly how to implement things ...


Welcome to the club! I am also thinking deeply about the best way of implementing (as is Ksilyan). A couple of things you might want to think about are:


  • How are you going to store data? One big file? Lots of small files?

    Here is one example of a problem area. I was initially planning to keep each player's data in its own file (like SMAUG does), but what happens if you want to (say) send mail to another player? Without his file loaded, you don't even know if he exists, or maybe does not accept mail from you, or is in the wrong faction, or something.

  • How are you planning to handle the difference between prototype objects (eg. the design for a short sword, or a level 5 naga), and the instances of them (eg. 100 players have the sword, there are 10 naga in a room)?

  • What, if any, event model do you want? For example, if I hit a naga with my sword. Who notifies other players that "Nick hits naga with sword"? The enactor (Nick)? The room? The naga? The player who is in the room with Nick? Maybe the act of hitting something raises an event, that other things in the room detect.



Good luck with the project, I hope you have lots of time on your hands. ;)
- Nick Gammon

www.gammon.com.au, www.mushclient.com
[Go to top] top
Posted by David Haley   USA  (3,881 posts)  [Biography] bio   Moderator
Date Wed 09 Aug 2006 05:00 PM (UTC)  [ quote  ]
Message
Life does have a tendency to do that. :-)

Sockets in C++ makes sense; I think having "systems-level" stuff be in C++ is generally a good idea. But even there, many scripting languages provide that functionality natively or with extensions. The more you put in C++, the more you'll have to port if/when you change hardware, operating systems, etc. The main advantage to a scripting package is that it hides a lot of that complexity. (Of course, C++ libraries also exist for this stuff...)


A common design is for things like AI to be scripted. So, every actor would have a script attached to it, with callbacks for various things. Think of it like mudprogs, actually: an NPC would have several functions attached that are called on certain triggers. You can use a model like this with a main loop in C++ or in the interpreted language.

Quote:
it seems that I will at least need to interface the socket handling with Ruby


That depends. In fact, you might not want to do that at all. Ruby could handle input strings, sure, but it wouldn't have to know anything about pulling bytes off of sockets. If you do the sockets in C++, you could raise an event (on the player's attached script, for example) whenever a line of input has been received.

Sorry for being a little brief at the moment; life has also caught up with me. :-) And after moving for the second half of my internship, I don't have internet access at home anymore. Grumble.
David Haley aka Ksilyan
Head Programmer,
Legends of the Darkstone

http://david.the-haleys.org
[Go to top] top
Posted by Nick Cash   USA  (626 posts)  [Biography] bio
Date Wed 09 Aug 2006 05:09 AM (UTC)  [ quote  ]
Message
Sorry for such a late reply! Life certainly picks up in a hurry.

The more I think about it the more it seems like doing a pure query interface would work best. It seems like interfacing the two languages would just be a hassle if I plan to make that type of move anyway.

That said, I am unsure exactly how to implement things (which is quite alright, as we are deep within the planning stage). We want the core functionality of the game to be in C++, mainly socket handling. Almost all other game logic would be handled in Ruby scripts.

What do you think would be the best implementation? It seems easy enough to call ruby and specify a script from C++, but that probably be quite as dynamic as we are looking for. On the other hand, we could probably embed a Ruby interpreter in the game core and run some sort of master Ruby script.

Regardless of what implementation is decided, it seems that I will at least need to interface the socket handling with Ruby.

Quote:

Quote:

That is, how could Python (or, in my case, Ruby) modify any actual game data in the C++ server? Do the script functions just call C++ equivalents that really do the work?


These are deep questions and I will respond in more detail tomorrow.


Any more details?

Thanks for the words of wisdom, as always.
~Nick Cash
http://www.nick-cash.com
[Go to top] top

The dates and times for posts above are shown in Universal Co-ordinated Time (UTC).

To show them in your local time you can join the forum, and then set the 'time correction' field in your profile to the number of hours difference between your location and UTC time.

12,248 views.

This is page 1, subject is 2 pages long: 1 2  [Next page]

[Reply to this subject]  Reply to this subject   [New subject]  Start a new subject   [Refresh] Refresh page

Go to topic:           Search the forum

[Go to top] top

[Home]

Written by Nick Gammon - 5K

Comments to: Gammon Software support
[RH click to get RSS URL] Forum RSS feed ( http://www.gammon.com.au/rss/forum.xml )

[Best viewed with any browser - 2K]    [Internet Contents Rating Association (ICRA) - 2K]    [Web site powered by FutureQuest.Net]