Notice: Any messages purporting to come from this site telling you that your password has expired, or that you need to verify your details, confirm your email, resolve issues, making threats, or asking for money, are
spam. We do not email users with any such messages. If you have lost your password you can obtain a new one by using the
password reset link.
Due to spam on this forum, all posts now need moderator approval.
Entire forum
➜ MUSHclient
➜ Suggestions
➜ Addition to serialize.lua - Functions
|
Addition to serialize.lua - Functions
|
It is now over 60 days since the last post. This thread is closed.
Refresh page
| Posted by
| Twisol
USA (2,257 posts) Bio
|
| Date
| Sun 14 Jun 2009 04:01 AM (UTC) Amended on Sun 14 Jun 2009 04:11 AM (UTC) by Twisol
|
| Message
| This is a suggested addition to the serialize.lua file, described and implmented here: http://www.gammon.com.au/forum/bbshowpost.php?id=4960
Fadedparadox and I were chatting a bit, and he had shown me his serialization functions to pack a Lua table into a MUSHclient variable. I noticed that it didn't handle functions, and he explained why. It stuck in my head though, I guess.
Today we were talking, and it came up again. I experimented a bit, dumping a basic function and iterating over each of its bytes (printing out the char number for each). The functions all have a lot of null bytes, so I figured that, since MUSHclient is written in C++, and cstrings are null-terminated, that's probably why storing/getting a dumped function doesn't work: it only stores "LuaQ".
A half-hour's pondering came up with a handy solution: encode the functions in base64 before storing them. I initially tried Base64Encode(), which didn't work because it was a C function exposed to Lua, so I wasted a few hours trying to write my own encoder/decoder. Then Fadedparadox pointed out utils.base64encode, heh.
In short: Given a function, get its string.dump representation, and use utils.base64encode on it to make it MUSH-variable friendly. To retrieve, use utils.base64decode and loadstring(). Example:
function foo()
return 42
end
encoded = utils.base64encode(string.dump(foo))
SetVariable("function_foo", encoded)
retrieved = GetVariable("function_foo")
newfoo = loadstring(utils.base64decode(retrieved))
Note(newfoo()) -- displays '42'
If this technique is used to store a function that's part of a table, you can build in the loadstring yourself, though obviously the user still has to loadstring the overall table:
out = [[ sometable = {
foo = loadstring(base64decode("]] .. utils.base64encode(string.dump(foo)) .. [["))
}]]
loadstring(out)()
Note(sometable.foo())
|
'Soludra' on Achaea
Blog: http://jonathan.com/
GitHub: http://github.com/Twisol | | Top |
|
| Posted by
| Fadedparadox
USA (91 posts) Bio
|
| Date
| Reply #1 on Sun 14 Jun 2009 07:52 AM (UTC) |
| Message
| As an example, this is my table save function I modified using Twisol's idea:
http://trevize.pastebin.com/m781a063
It seems to work perfectly.
*thumbsup twisol* | | Top |
|
| Posted by
| Nick Gammon
Australia (23,140 posts) Bio
Forum Administrator |
| Date
| Reply #2 on Sun 14 Jun 2009 09:33 PM (UTC) |
| Message
| Why would you want to do this? Variables change (eg. hp goes up), but functions don't, as they are written by a human.
I know some Lua functions "generate" other functions, which is effectively done by adding local variables to them (ie. upvalues). However the documentation for string.dump says "The function must be a Lua function without upvalues.".
This idea may work in the short term, but if you serialize stuff which is saved to disk for use later, and Lua has a version upgrade, the serialized function may not work in the new version. |
- Nick Gammon
www.gammon.com.au, www.mushclient.com | | Top |
|
| Posted by
| Twisol
USA (2,257 posts) Bio
|
| Date
| Reply #3 on Sun 14 Jun 2009 10:07 PM (UTC) Amended on Sun 14 Jun 2009 10:25 PM (UTC) by Twisol
|
| Message
| This would be nice for completeness' sake, for one. For another, I have a relatively contrived example:
If you have a table containing two numeric values, and a variable to store a function, you could assign functions to that variable which would have different effects on the values. For example, add, subtract, exponentiation, etc. When you serialize the table (plus function), it remembers what "mode" it was meant to be in. In this case, upvalues wouldn't be a problem.
Here's some example code that I tested (also rather contrived), though it doesn't actually make use of the scenario above. It just shows how upvalues aren't a problem here.
foo = function()
return sometable.x + sometable.y
end
out = [[ sometable = {
x = 1,
y = 2,
foo = loadstring(utils.base64decode("]] .. utils.base64encode(string.dump(foo)) .. [["))
]]
loadstring(out)()
Note(sometable.foo())
EDIT: Possible example of the usefulness of serializing a function, completely apart from the table serialization bit, is that it can be sent to another plugin to modify its behavior, if it's built for such tinkering.
EDIT 2: For further clarification, you don't need to use a function that generates a function for this to be of use. It's basically just changing an algorithm that the object uses, and persisting that over saves. |
'Soludra' on Achaea
Blog: http://jonathan.com/
GitHub: http://github.com/Twisol | | Top |
|
| Posted by
| WillFa
USA (525 posts) Bio
|
| Date
| Reply #4 on Sun 14 Jun 2009 11:30 PM (UTC) |
| Message
| Well, I can see good reasons for it, and a bad thing about it...
Good: If you use Tables as objects, you can persist the whole object, instead of designing the data segment to be separate and saving just that.
Bad: Distributing base64 encoded bytecode means you better be damn sure you trust whoever is giving you plugins. You lose the transparency of having functions in plain text so people can reassure themselves you're not trying to do Bad Things(tm) to them.
Bad2: Encoding multiple copies of the same function in different variables bloats the world/state file. These values aren't as small as just a simple string or integer after all.
Good2: I can see passing functions encoded to another plugin, that can be added to an array to simulate your own plugin callbacks... Just in case you like over engineering solutions and are afraid there'll be some random plugin that you didn't code (thus, obviously coded wrong :) ) that doesn't check source or message number or something and responds to EVERY broadcast...
| | Top |
|
| Posted by
| Nick Gammon
Australia (23,140 posts) Bio
Forum Administrator |
| Date
| Reply #5 on Sun 14 Jun 2009 11:31 PM (UTC) |
| Message
| I suppose, but the whole idea of a scripting language is it is high-level. Serializing functions is really serializing the internal byte-codes used by the Lua interpreter. It just seems to me this is going against the grain of a high-level scripting language.
Of course, you are welcome to do this in your own scripts, but I think to add it into the stock distribution might imply functionality that isn't really there (eg. the upvalues, problems with future versions, etc.). |
- Nick Gammon
www.gammon.com.au, www.mushclient.com | | Top |
|
| Posted by
| Twisol
USA (2,257 posts) Bio
|
| Date
| Reply #6 on Sun 14 Jun 2009 11:48 PM (UTC) Amended on Sun 14 Jun 2009 11:50 PM (UTC) by Twisol
|
| Message
| @ WillFa: For Bad #1, that's true, there's a touchy side of this I hadn't considered. But you can do a lot of bad even without this "sneaky" bas64 thing, and it certainly wouldn't need to be added to serialize.lua to happen. Plus, I'm certain that a compiled .lua file could be supplied with a plugin, and all that's needed to use it is to dofile() it. There's absolutely no base64 there, and the point of the compiled Lua is hidden from sight. That's not something my technique creates: it's there already.
For Bad #2, that's also true, but consider that most functions wouldn't need serialization. This is a niche technique that is definitely useful where needed, but can usually be left alone. In my opinion, it's good for completeness' sake.
For Good #2, that's true, except I'd much rather use OnPluginBroadcast or CallPlugin anyways; using serialized functions seems hacky in this case.
@ Nick: When you're interacting with C++ via a Lua interface, I think it becomes slightly less high-level. Consider: if all we ever used here was pure Lua, then there would be no need for the obfuscation of the dumped function. It's only against the grain because it's a necessary evil.
Also, being able to serialize a function and save it somewhere is already supported by Lua by virtue of string.dump(), and this sort of thing could be emulated (albeit roughly) by saving the serialized functions to a file, and calling another plugin with the name of the file to load. It's, again, a bit hacky, so adding in-built support for serialized functions - which isn't terribly hard as I've displayed - simply makes it easier to do.
Upvalues are only a problem with closure functions so far as I've seen, and you can serialize the closure function itself, just not the function it creates (so far as I know). With future versions... you always have those kinds of problems, with loss of backwards compatability. I don't think high-value functions will be stored this way, though -and if they are, it's either fleetingly for passing it elsewhere within the client, or the coder better know the inherent risk.
EDIT: Anyways, thanks for the comments. At the very least it helped me understand some of the implications of this technique, and if I find a bona fide use for it, you can bet I'll be using it. Something to add to my "Oh nifty" list! |
'Soludra' on Achaea
Blog: http://jonathan.com/
GitHub: http://github.com/Twisol | | Top |
|
The dates and times for posts above are shown in Universal Co-ordinated Time (UTC).
To show them in your local time you can join the forum, and then set the 'time correction' field in your profile to the number of hours difference between your location and UTC time.
26,222 views.
It is now over 60 days since the last post. This thread is closed.
Refresh page
![[Go to top]](/images/gototop.gif)
top