[Home] [Downloads] [Search] [Help/forum]

Gammon Software Solutions forum

See www.mushclient.com/spam for dealing with forum spam. Please read the MUSHclient FAQ!

[Folder]  Entire forum
-> [Folder]  MUSHclient
. -> [Folder]  Lua
. . -> [Subject]  String.find not working for regex strings in table
Home  |  Users  |  Search  |  FAQ
Username:
Register forum user name
Password:
Forgotten password?
String.find not working for regex strings in table

[Reply to this subject]  Reply to this subject   [New subject]  Start a new subject   [Refresh] Refresh page

Posted by Natasi   (79 posts)  [Biography] bio
Date Wed 23 May 2012 02:00 AM (UTC)  [ quote  ]
Message
I have a table (example below) filled with about 700 entries, which I go through using string.find and compare to the incoming trigger. Issue is, string.find is not working with any regex in the line, it will only match up on on plain text matches.

triggerstext = {}
triggerstext["Your mouth turns up as your face (.*?) into a smile."]=function() Note("Look mom, I found you!"); end

Here is the function I parse the trigger with:

function parseAll(label, trigger, wildcard)

for _, trig in pairs (triggerstext) do
 a = string.find(trigger, _);
 if a then
   trig();
 end
end

end

So the above will only ever match on plain, not on the regex portion. I have also written this as a regex function using, but it is extremely slow when going over 700 entries in the table :

for _, trig in pairs (triggerstext) do
re = rex.new (_);
a, b, c = re:match (trigger);
if a then
trig(c); --fires function
end
end

Any help with the string.find issue would be appreciated, or if anyone knows why the regex function is so insanely slow, that would help also. My LUA is not advanced enough yet to figure this out so far.
[Go to top] top
Posted by Fiendish   USA  (848 posts)  [Biography] bio   Global Moderator
Date Reply #1 on Wed 23 May 2012 02:42 AM (UTC)  [ quote  ]
Message
Lua does not use regex as part of the language. Lua uses its own pattern format described here: http://www.lua.org/pil/20.2.html

You probably want to precompile your rexes instead of doing it again 700 times every line.
http://aardwolfclientpackage.googlecode.com/
[Go to top] top
Posted by Natasi   (79 posts)  [Biography] bio
Date Reply #2 on Wed 23 May 2012 04:32 AM (UTC)  [ quote  ]
Message
Ahhhh I was using (.*?) and should have been using (.*)... that fixed that issue, thanks!

As for the re:match one precompile, I tried that using the below method and it was still very slow. On load of the script I would precompile all the triggers then go through them on trigger.

function preParse()
i = 1;
for _, trig in pairs (plaintext) do
_G ["re_"..i] = rex.new (_);
i= i+1;
end
end


function parseAll1(label, trigger, wildcard)
tt = 1
for _, trig in pairs (plaintext) do
a, b, c = _G["re_"..tt]:match (trigger);
if a then
trig(c);
end
tt = tt+1;
end
end


Is there a better way of writing this?
[Go to top] top
Posted by Nick Gammon   Australia  (18,769 posts)  [Biography] bio   Forum Administrator
Date Reply #3 on Wed 23 May 2012 05:45 AM (UTC)  [ quote  ]
Message
Your original one failed because the Lua regex does not recognize (.*?) syntax. This works:


triggerstext = {}
triggerstext["Your mouth turns up as your face (.*) into a smile."]=function() Note("Look mom, I found you!"); end

function parseAll(label, trigger, wildcard)

  for pattern, trig in pairs (triggerstext) do
   if string.find (trigger, pattern) then
      trig ()
      break
   end
  end

end

local start = utils.timer ()
parseAll ("foo", "Your mouth turns up as your face Nick into a smile.")
print (utils.timer () - start)



Output:


Look mom, I found you!
0.00070651416899636


So under a millisecond for detecting that. I added the "break" because once you get a match you don't need to keep trying.

Most of that time would have been processing the match, because if I send in a non-matching string it only took about 7 microseconds.
- Nick Gammon

www.gammon.com.au, www.mushclient.com
[Go to top] top
Posted by Nick Gammon   Australia  (18,769 posts)  [Biography] bio   Forum Administrator
Date Reply #4 on Wed 23 May 2012 05:56 AM (UTC)  [ quote  ]
Message
What would speed this up, too, would be to have this trigger (which does the 700 matches) be a low-priority trigger, down the list in your trigger list behind other "more likely to match" triggers. For example, a prompt. So if you get a prompt your prompt trigger fires first, which saves even testing a single item in this table.
- Nick Gammon

www.gammon.com.au, www.mushclient.com
[Go to top] top
Posted by Nick Gammon   Australia  (18,769 posts)  [Biography] bio   Forum Administrator
Date Reply #5 on Wed 23 May 2012 05:57 AM (UTC)  [ quote  ]

Amended on Wed 23 May 2012 05:58 AM (UTC) by Nick Gammon

Message
Look up string.find in the inbuilt help to find the exact Lua regexp syntax. The Lua regexp is quite fast, but its syntax differs a bit to the one used by triggers.

This is the web version (you get the same stuff in the help):

http://www.gammon.com.au/scripts/doc.php?lua=string.find
- Nick Gammon

www.gammon.com.au, www.mushclient.com
[Go to top] top
Posted by Natasi   (79 posts)  [Biography] bio
Date Reply #6 on Wed 23 May 2012 11:26 AM (UTC)  [ quote  ]
Message
If my triggertext had multiple wildcards in it, how would I capture those?

In the code below, I would want to see the first wildcard output, which would be 'up', and the second, which would be 'breaks'. I did a quick version doing a, b, c = string.find() and I could get the 'up' by outputting c, but unless I added a d, e, f (etc) it wouldn't catch additional wildcards.

triggerstext = {}
triggerstext["Your mouth turns (.*) as your face (.*) into a smile."]=function() Note("Look mom, I found you!"); end

function parseAll(label, trigger, wildcard)

for pattern, trig in pairs (triggerstext) do
if string.find (trigger, pattern) then
trig ()
break
end
end

end

local start = utils.timer ()
parseAll ("foo", "Your mouth turns up as your face Nick into a smile.")
print (utils.timer () - start)
[Go to top] top
Posted by Natasi   (79 posts)  [Biography] bio
Date Reply #7 on Wed 23 May 2012 11:34 AM (UTC)  [ quote  ]
Message
Another issue I've come across, it seems (wildcard1|wildcard2) splits do not work with Lua string.find. Would it be wise to split all my triggers apart (I have about 1284 '|' grouped triggers) into their own individual lines or try to find a way to speed up the regex version of this?
[Go to top] top
Posted by Nick Gammon   Australia  (18,769 posts)  [Biography] bio   Forum Administrator
Date Reply #8 on Wed 23 May 2012 11:59 AM (UTC)  [ quote  ]
Message
Natasi said:

If my triggertext had multiple wildcards in it, how would I capture those?


Like this:


triggerstext = {}
triggerstext["Your mouth turns (.*) as your face (.*) into a smile."]
             =
             function(where, who) 
               Note("Look " .. who .. ", I found you! and your face turned " .. where); 
             end

function parseAll(label, trigger, wildcard)

  for pattern, trig in pairs (triggerstext) do
    local results = { string.match (trigger, pattern) }
    if next (results) then
      trig (unpack (results))
      break
   end
  end

end

local start = utils.timer ()
parseAll ("foo", "Your mouth turns up as your face Nick into a smile.")
print ((utils.timer () - start) )


Output:


Look Nick, I found you! and your face turned up
0.00056068558478728



Quote:

but it is extremely slow when going over 700 entries in the table :


...

Quote:

I have about 1284 '|' grouped triggers


They seem to be multiplying!

Yes, the "|" operator does not work in Lua regexps.
- Nick Gammon

www.gammon.com.au, www.mushclient.com
[Go to top] top
Posted by Natasi   (79 posts)  [Biography] bio
Date Reply #9 on Wed 23 May 2012 04:03 PM (UTC)  [ quote  ]
Message
Thanks Nick!

The 700 entries referred to this:

triggerstext["Your mouth turns (.*) as your face (.*) into a smile."]...



The 1284 refers to the multiple within:

triggerstext["Your mouth turns (.*) as your face (.*) into a smile.|I like pie.|This is another one to match"]...


So I take it I just have to split them apart then, just to be sure, the string.math/find over the soon to be almost 2000 triggers will still parse faster than the regex over 700?
[Go to top] top
Posted by Worstje   Netherlands  (867 posts)  [Biography] bio
Date Reply #10 on Wed 23 May 2012 04:13 PM (UTC)  [ quote  ]
Message
Natasi, it very likely will. The reason is that the complexity is less: PCRE has tons of options but like a car, the extra weight also carries its toll. Lua has less features and is thus less complex, meaning it can generally execute equivalent patterns faster.

Also, I have experience using several thousand builtin MUSHclient triggers myself, and while it lagged things just a little it was still something I didn't need to pay much heed to. This was 4+ years ago, so I imagine performance has only increased since.
[Go to top] top
Posted by Nick Gammon   Australia  (18,769 posts)  [Biography] bio   Forum Administrator
Date Reply #11 on Thu 24 May 2012 02:00 AM (UTC)  [ quote  ]
Message
It seems like a lot, but yes you will have to live without the "|" if you are going to use Lua.

I suggest you "anchor" them, that should make it much faster:


"^Your mouth turns (.*) as your face (.*) into a smile.$"


The reason is, without the anchor, say it gets:


There is a shop here.


Now it compares the regexp "Y" (as "Your mouth") to "T" (as in "There is a shop here."). It finds no match. But without the anchor, it then moves in one character and tries again. After all the line might say: 


There is a shop here. Your mouth turns blue as your face crinkles into a smile.


It gets no match on the "h" and tries again. And again. And again.

But with the anchor (the "^" character) if it doesn't match at the start of the line it stops immediately.

If that isn't fast enough, you could split them up a bit.

For example, all the triggers starting "You" could be put into a group (separate table). Then a single test (does the line start with "You"?) could let you decide whether to test a further 100 triggers or no.
- Nick Gammon

www.gammon.com.au, www.mushclient.com
[Go to top] top

The dates and times for posts above are shown in Universal Co-ordinated Time (UTC).

To show them in your local time you can join the forum, and then set the 'time correction' field in your profile to the number of hours difference between your location and UTC time.

1,351 views.

[Reply to this subject]  Reply to this subject   [New subject]  Start a new subject   [Refresh] Refresh page

Go to topic:           Search the forum

[Go to top] top

[Home]

Written by Nick Gammon - 5K

Comments to: Gammon Software support
[RH click to get RSS URL] Forum RSS feed ( http://www.gammon.com.au/rss/forum.xml )

[Best viewed with any browser - 2K]    [Internet Contents Rating Association (ICRA) - 2K]    [Web site powered by FutureQuest.Net]