Register forum user name Search FAQ

Gammon Forum

Notice: Any messages purporting to come from this site telling you that your password has expired, or that you need to verify your details, confirm your email, resolve issues, making threats, or asking for money, are spam. We do not email users with any such messages. If you have lost your password you can obtain a new one by using the password reset link.

Due to spam on this forum, all posts now need moderator approval.

 Entire forum ➜ MUSHclient ➜ General ➜ Freezing Mushclient

Freezing Mushclient

It is now over 60 days since the last post. This thread is closed.     Refresh page


Pages: 1  2 

Posted by Worstje   Netherlands  (899 posts)  Bio
Date Reply #15 on Fri 27 Jul 2007 10:31 AM (UTC)
Message
Definately. With send to script, MUSHclient needs to parse the stuff in the Send field each time the trigger is fired (to take care of %1 and all that). After that, the code is being fed to the script engine of your world. Then the script engine has to translate it again.

Using send to script, the code is only read once and then stays within the script engine. MUSH simply tells the script engine 'execute function PromptCapture', which is definately less overhead.
Top

Posted by Nick Gammon   Australia  (23,133 posts)  Bio   Forum Administrator
Date Reply #16 on Fri 27 Jul 2007 09:21 PM (UTC)
Message
Quote:

Using send to script, the code is only read once ...


You mean, using the script file.

I am glad that solves the problem. "Send to script" shouldn't really consume lots of memory, but it sounds like Python is using something internally that it doesn't release.

Worstje is right, using a script file will incur less overhead. Doing "send to script" in triggers (etc.) is easier because you can see the code in front of your eyes, but less efficient.

- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top

Posted by Isthiriel   (113 posts)  Bio
Date Reply #17 on Fri 03 Aug 2007 09:21 AM (UTC)
Message
I have a similar problem with my MushClient/Python, though I don't use Send To Script at all; everything is script calls.

Each function call on the script adds about 4KB to MushClient's footprint. Python itself doesn't leak memory in my experience. I use it for automation all the time.

However, Python's garbage collector is a refcounter, not mark-and-sweep. So it can't detect cycles.

My assumption has been that the World object handed to Python is not being destructed properly because it contains a cycle.
Top

Posted by Nick Gammon   Australia  (23,133 posts)  Bio   Forum Administrator
Date Reply #18 on Fri 03 Aug 2007 11:30 AM (UTC)
Message
I don't create the world object each time, so I don't see how MUSHclient is doing it. If there is a fundamental problem, since all script engines, except Lua, are called using the Windows Script Interface, all languages would have a leak.

- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top

Posted by Isthiriel   (113 posts)  Bio
Date Reply #19 on Fri 03 Aug 2007 04:43 PM (UTC)
Message
Lua's garbage collector is mark-and-sweep.

AFAIK, so is Javascript's, VBScript's and PerlScript's.

If the world object is being handed to Python with a cycle and on each script invocation the world object doesn't reuse the exact same Python objects... I think that's enough to make it leak.

One of these days I'll open up the MushClient source and see if I can work out what the problem is. It's definitely an integration/interaction problem rather than something that can be laid at the feet of either side :(
Top

Posted by Nick Gammon   Australia  (23,133 posts)  Bio   Forum Administrator
Date Reply #20 on Sat 04 Aug 2007 12:40 AM (UTC)
Message
I tried running this small script:


for i = 1, 1000 do 
  GetInfo (1)
end -- for


This effectively calls a function 1000 times, but when using Lua and doing this, the memory footprint seemed to rise about 40 Kb.

Doing this under VBscript (which uses COM) seemed to consume about 24 Kb. This is a lot less than then 4 Kb "per call" you reported.

You could probably make a case that the memory should not rise at all, so there may be a problem here.

In the case of GetInfo (1) a string is being created, so perhaps it is not being disposed of correctly. Looking at what is happening here, GetInfo (1) does this:


VARIANT CMUSHclientDoc::GetInfo(long InfoType) 
{
VARIANT vaResult;
VariantInit(&vaResult);

  vaResult.vt = VT_EMPTY;   // in case dates are zero

  switch (InfoType)
    {
    case    1: SetUpVariantString (vaResult, m_server); break;

// blah blah for other cases

    default:
      vaResult.vt = VT_NULL;
      break;

    } // end of switch

	return vaResult;
}  // GetInfo


And SetUpVariantString does this:


void CMUSHclientDoc::SetUpVariantString (VARIANT & tVariant, const CString & strContents)
  {
  VariantClear (&tVariant);
  tVariant.vt = VT_BSTR;
  tVariant.bstrVal = strContents.AllocSysString (); 
  }   // end of CMUSHclientDoc::SetUpVariantString


So, clearly a string is being allocated, and the allocation of around 1000 strings might account for a lot of the memory usage I am seeing.

However as that string is being passed back to the script engine (the one that did the GetInfo) it seems to me that the script engine is responsible for freeing the string up when it is done with it.

- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top

Posted by Isthiriel   (113 posts)  Bio
Date Reply #21 on Sun 05 Aug 2007 12:15 PM (UTC)

Amended on Sun 05 Aug 2007 03:38 PM (UTC) by Isthiriel

Message
Ah, I meant for every call from MushClient to Python. And I got the 4KB number from running a 0.1s timer for a minute and noticing the memory footprint increased by about 2.5MB

I added:
def test_memory():
    for i in range(1000):
        world.GetInfo(1)


to my script file and executed it. The first run seemed to add 4KB. Subsequent runs added 0KB to the footprint. (I should note that 4KB seems to be Task Manager's minimum granularity for memory usage.)

Recompiling the script randomly added from 20KB to 280KB :( Which smells like a bug in the WSH interface that the pywin32 extension implements.



Checking the result I mentioned in the first paragraph I watched the footprint increase by 3.8MB over a minute ... and then noticed that the timer was doing a Send-to-Script rather than a straight function call (because I defined my poll() function without arguments in the script, *sigh*). Changing it to a function call stopped the leak (memory usage stable at 28636K and script calls from timer > 4000 while I typed this post).

HTH.

EDIT: (I'm using Python 2.5.1 [2007-Apr-18] and pywin32 build 210 [2006-Sep-22], which are the latest versions of each.)

EDIT[2]: (Python 2.5 because I'm a neophile and the "with" statement.)
Top

Posted by Worstje   Netherlands  (899 posts)  Bio
Date Reply #22 on Sun 05 Aug 2007 01:29 PM (UTC)

Amended on Sun 05 Aug 2007 01:32 PM (UTC) by Worstje

Message
First of all, oops. I did mean script-file in that post far above. (Edit: I keep writing down the wrong one, bad addiction, gah!)

Second, I don't have too much experience with Python globbing memory like an insane beast. I have a plugin with 650 triggers in it (and growing), and I can reload it to my hearts content. While loading, it temporarily jumps up by around 1mb in memory, but immediately drops again. I tried kicking different triggers into action (both send to script ones and script-file variety), and I can easily say that neither makes a difference for me.

Then again; this computer uses a pretty old version of Python (2.4.2) and a pretty old version of pywin32. Maybe it is a more recent bug in either of those that is causing it?
Top

The dates and times for posts above are shown in Universal Co-ordinated Time (UTC).

To show them in your local time you can join the forum, and then set the 'time correction' field in your profile to the number of hours difference between your location and UTC time.


68,791 views.

This is page 2, subject is 2 pages long:  [Previous page]  1  2 

It is now over 60 days since the last post. This thread is closed.     Refresh page

Go to topic:           Search the forum


[Go to top] top

Information and images on this site are licensed under the Creative Commons Attribution 3.0 Australia License unless stated otherwise.