Lua tables in detail

Posted by Nick Gammon on Sat 29 Oct 2005 09:17 PM — 11 posts, 102,830 views.

Nick Gammon Australia Forum Administrator Sat 29 Oct 2005 09:17 PM #0

What are tables?

Tables are key/value pairs, designed for fast lookup of the value corresponding to a particular key.

What data can you put in tables?

Any Lua types, excepting the special "non-value" nil, can be used for either the key or the value part of tables.

tprint function

For illustrative purposes I will be using the "tprint" function I wrote a while ago that recursively shows the contents of a table. This function ships with MUSHclient in the tprint.lua file. To use it, simply put this line near the start of your script:


require "tprint"

Table constructors

Before you can use a table it must be "constructed" using a variant of the syntax:



t = { }

The above constructs an empty table and puts a reference to it in variable "t".


t = {
  ["name"] = "nick",
  ["class"] = "mage",
  ["hp"] = 234,
  ["online"] = true
}

tprint (t)

Output

"online"=true
"name"="nick"
"class"="mage"
"hp"=234

Immediately we notice something interesting, the order in which items are retrieved from a table is not necessarily the order in which they were inserted, nor alphabetic order either. Since tables are stored internally using "hashing" algorithms there is no guarantee of which order items are retrieved, when doing a linear scan. The exception to this is using numeric keys to simulate a vector, described further on.

The above example can be rewritten without the quotes and square brackets, as a form of shorthand, which has the same result, if you wish to key by string keys:


t = {
  name = "nick",
  class = "mage",
  hp = 234,
  online = true
}

tprint (t)

Output

"online"=true
"name"="nick"
"class"="mage"
"hp"=234

The shorthand method shown above, where you omit the square brackets, only applies if the key is a valid Lua dataname (ie. starting with a letter or underscore, and then consisting of letters, numbers and underscores).

You can also use any other data type for keys, except nil. Usually this means using numeric keys, but you are not restricted to that.


t = {
  [10] = "number 10",
  ["10"] = "string 10",
  [true] = "a true key",
  [print] = "keyed by a function"
}

tprint (t)

Output

10="number 10"
true="a true key"
function: 00DD6020="keyed by a function"
"10"="string 10"

Note an important point here, the item keyed by the number 10 is a different item to that keyed by the string "10".

You can let Lua assign numeric keys for you, especially if you are trying to set up an "ordered" table, like this:


t = {
    "the", 
    "quick", 
    "brown", 
    "fox", 
    "jumped"
}

tprint (t)

Output

1="the"
2="quick"
3="brown"
4="fox"
5="jumped"

Here Lua has assigned a numeric key to each item, starting at 1.

You can use variables as keys in which case the underlying value is used:



a = "nick"
b = 22
c = print

t = {
  [a] = "fish",
  [b] = "mage",
  [c] = 234,
}

tprint (t)

Output

function: 00DD3070=234
"nick"="fish"
22="mage"

An important distinction is between these two forms:



t [a] = "something"  --> indexed by whatever variable 'a' contains
t ["a"] = "something else" --> indexed literally by the string "a"

Accessing table elements

Once your table has been created you can access individual elements by referring to their key:


t = {
  ["name"] = "nick",
  ["class"] = "mage",
  ["hp"] = 234,
  ["online"] = true,
  [99] = "a number"
}

print (t.name) --> nick
print (t ["class"]) --> mage
print (t [99]) --> a number

Once again you refer to the key by putting it in square brackets (and then quoting it if it is a string), however as a shorthand you can simply have tablename.key providing the key is a valid Lua identifier (as in the case of t.name above).

Changing table elements

You can change individual items, or add more, using similar syntax:


t = {
  ["name"] = "nick",
  ["class"] = "mage",
  ["hp"] = 234,
  ["online"] = true,
  [99] = "a number"
}

t.name = "john"
t ["class"] = "warrior"
t [99] = 88
t.race = "dwarf"

tprint (t)

Output

"online"=true
"name"="john"
99=88
"class"="warrior"
"hp"=234
"race"="dwarf"

The above example adds a new element (t.race) by simply assigning to it. If the entry does not exist it is created.

Deleting table elements

You delete a table item by assigning nil to it:


t = {
  ["name"] = "nick",
  ["class"] = "mage",
  ["hp"] = 234,
  ["online"] = true,
  [99] = "a number"
}

t.name = nil
t ["class"] = nil

tprint (t)

Output

"online"=true
99="a number"
"hp"=234

Scanning through a table

Often you want to find everything in a table. There are two main ways of doing this. The simplest is to use table.foreach, like this:


t = {
  name = "nick",
  class = "mage",
  hp = 234,
  online = true
}

table.foreach (t, print)

Output

online true
name nick
class mage
hp 234

The function table.foreach iterates over a table, calling the supplied function with two arguments for each table entry, the corresponding key and value. You can write your own function of course:


t = {
  name = "nick",
  class = "mage",
  hp = 234,
  online = true
}

function f (k, v)
  print ("key = " .. tostring (k) .. 
         ", value = " .. tostring (v))
end -- f

table.foreach (t, f)

Output

key = online, value = true
key = name, value = nick
key = class, value = mage
key = hp, value = 234

You have the option in your function of returning a non-nil value based on some decision, in which case the iterating of the table stops, and the value you supply is returned as the return value of the foreach function...


t = {
  name = "nick",
  class = "mage",
  hp = 234,
  online = true
}

function f (k, v)
  print ("key = " .. tostring (k) .. ", value = " .. tostring (v))
  if v == "nick" then
    return true
  end -- if
end -- f

table.foreach (t, f)


Output

key = online, value = true
key = name, value = nick

The other main way of iterating through a table is to use the "for ... pairs" construct, like this:


t = {
  name = "nick",
  class = "mage",
  hp = 234,
  online = true
}

for k, v in pairs (t) do
  print (k, v)
end -- for loop

Output

online true
name nick
class mage
hp 234

This method is a bit more flexible as it lets you do things inside your loop (like recursively loop like the tprint function does).

There are variants on these two methods ('table.foreachi', and 'for ... ipairs') which are discussed further down under the treatment of ordered tables (vectors).

Tables can contain tables

Since we have seen that tables can hold any data type, except nil, then clearly tables can themselves consist of sub-tables.


t = { 

  dwarf = { str = 22, dex = 23, wis = 18 },
  human = { str = 20, dex = 20, wis = 20 },
  elf   = { str = 18, dex = 24, wis = 25 },

}

tprint (t)

Output

"dwarf":
  "str"=22
  "dex"=23
  "wis"=18
"human":
  "str"=20
  "dex"=20
  "wis"=20
"elf":
  "str"=18
  "dex"=24
  "wis"=25

The tprint function helps to see the nested relationship by indenting sub-tables.

Tables are anonymous and stored by reference

A table is not tied to a particular dataname (variable). For a start, you can have tables without any name:


tprint ( { "the", "quick", food = "eggs" } )

tprint (t)

Output

1="the"
2="quick"
"food"="eggs"

In the above example we are printing a table constructed on-the-fly which has no name.


t1 = { "the", "quick", food = "eggs" } 
t2 = t1   --> copy reference to table
t2.food = "fish"

tprint (t1)
print "----"
tprint (t2)

Output

1="the"
2="quick"
"food"="fish"
----
1="the"
2="quick"
"food"="fish"

Here we copied the reference to the table from t1 to t2, however they both point to the same table. Hence when I changed t2.food to "fish", it also got changed in t1.

Deleting tables

You do not explicitly delete tables. Once there are no references to a table the Lua garbage-collector will eventually delete it.

Using foreach to find a particular entry

You can use a function combined with foreach to do a linear search of a table until a certain condition is satisfied. Here, let's find which race has a wisdom of 25 (the first one, anyway) ...


t = { 

  dwarf = { str = 22, dex = 23, wis = 18 },
  human = { str = 20, dex = 20, wis = 20 },
  elf   = { str = 18, dex = 24, wis = 25 },

}

print (table.foreach (t, 
      function (k, v)
       if v.wis == 25 then
         return k
       end -- if 
      end -- function
      ))

Output

elf

Nick Gammon Australia Forum Administrator Sat 29 Oct 2005 10:51 PM #1

Tables as vectors

As noted above, normal tables do not guarantee any particular order for returning their entries, using a linear scan (table.foreach).

However often we want to keep our data in some kind of sequence. Lua provides for that by the concept of using numeric keys for tables, and assuming that you can scan through a table by starting at key 1, and progressing through it until you reach the 'size' of the table.

Let's look at a table with a combination of numeric and string keys:


t = 
  {
  "the", 
  "quick", 
  "brown", 
  "fox", 
  "jumped",
  hp = 22,
  mana = 44,
  skill = 80
}

table.foreach (t, print)

Output

1 the
2 quick
3 brown
4 fox
5 jumped
mana 44
hp 22
skill 80

As it turns out, the numeric keys have been displayed in sequence, but this is not necessarily going to be the case. Now let's print using 'foreachi' which simply iterates over the numeric keys:


table.foreachi (t, print)

Output

1 the
2 quick
3 brown
4 fox
5 jumped

Let's add item 7 manually and see what happens:


t [7] = "test"

table.foreachi (t, print)

Output

1 the
2 quick
3 brown
4 fox
5 jumped

It still thinks there are only 5 items, because of the gap between 5 and 7. Let's put item 6 in ...


t [6] = "missing link"

table.foreachi (t, print)

Output

1 the
2 quick
3 brown
4 fox
5 jumped
6 missing link
7 test

Now we see items 1 to 7. It is pretty clear how Lua is working internally. To go through a table sequentially it simply looks up key 1, then key 2, and so on, until it finds a gap, then it stops.

(Caveat - actually it sequences from 1 to the 'table size' which is something you can control. The default unless you start mucking around is the highest numeric key in sequence).

The other way we can go through the numeric items is to use 'for ... ipairs' like this:


for k, v in ipairs (t) do
  print (k, v)
end -- for

Output

1 the
2 quick
3 brown
4 fox
5 jumped
6 missing link
7 test

In this case the 'ipairs' iterator only looks for numeric entries, starting at 1, rather than all table entries, which is what 'pairs' returns.

Inserting into a vector

A simple way of adding a numeric item to a table is by using table.insert, like this:


table.insert (t, "a new entry")

table.foreachi (t, print)

Output

1 the
2 quick
3 brown
4 fox
5 jumped
6 missing link
7 test
8 a new entry

This function works out where the table ends, and appends a new entry to the end. You can also insert by specifying a position:


table.insert (t, 5, "trouble at mill")

table.foreachi (t, print)

Output

1 the
2 quick
3 brown
4 fox
5 trouble at mill
6 jumped
7 missing link
8 test
9 a new entry

This has made entry 5 into the new entry, and pushed all the others down by 1.

Removing items from a vector

The simple case lets you remove the last entry:


t = 
  {
  "the", 
  "quick", 
  "brown", 
  "fox", 
  "jumped",
}

table.remove (t)

tprint (t)

Output

1="the"
2="quick"
3="brown"
4="fox"

You can specify which entry to remove:


t = 
  {
  "the", 
  "quick", 
  "brown", 
  "fox", 
  "jumped",
}

table.remove (t, 3)

tprint (t)
Output

1="the"
2="quick"
3="fox"
4="jumped"

In this case it had to resequence each item past the deleted one, so each entry is still in sequence.

Sorting a table

If you sort a table, it resequences the keys, based on the comparison operation you give it. The default is to compare the values for "less than".


t = 
  {
  "the", 
  "quick", 
  "brown", 
  "fox", 
  "jumped",
}

table.sort (t)

tprint (t)

Output

1="brown"
2="fox"
3="jumped"
4="quick"
5="the"

Only the "vector" part of the table is sorted, the rest (if any) is left alone:


t = 
  {
  "the", 
  "quick", 
  "brown", 
  "fox", 
  "jumped",
  hp = 22,
  mana = 44,
  skill = 80
}
table.sort (t)
tprint (t)

Output

1="brown"
2="fox"
3="jumped"
4="quick"
5="the"
"mana"=44
"hp"=22
"skill"=80

Specifying a sort sequence

In the case of simple values the default "less than" comparison may be fine, however some cases need an explicit comparison:


t = { 

   { str = 22, dex = 23, wis = 18 },
   { str = 20, dex = 20, wis = 20 },
   { str = 18, dex = 24, wis = 25 },

}

table.sort (t) --> error: attempt to compare two table values

So, we supply a comparison function, and inside that choose which field to compare with. I will use the "str" field of the subtable, to sort them into "str" order ...


t = { 

   { str = 22, dex = 23, wis = 18 },
   { str = 20, dex = 20, wis = 20 },
   { str = 18, dex = 24, wis = 25 },

}

table.sort (t, 
           function (v1, v2)
             return v1.str < v2.str
           end -- function
           )
tprint (t)


Output

1:
  "str"=18
  "dex"=24
  "wis"=25
2:
  "str"=20
  "dex"=20
  "wis"=20
3:
  "str"=22
  "dex"=23
  "wis"=18

We can see here that the table is now sorted into "str" order of the subtable entries.

Sorting the key

But how would we sort an alphabetic key, you might ask? Take for example this table:


t = { 

  dwarf = { str = 22, dex = 23, wis = 18 },
  human = { str = 20, dex = 20, wis = 20 },
  elf   = { str = 18, dex = 24, wis = 25 },

}

How can we get that into alphabetic order by key? The trick is to use a second table, that refers to elements in the first one.

Here is one approach:


t = { 

  dwarf = { str = 22, dex = 23, wis = 18 },
  human = { str = 20, dex = 20, wis = 20 },
  elf   = { str = 18, dex = 24, wis = 25 },

}

t2 = {}

table.foreach (t, function (k) table.insert (t2, k) end )

table.sort (t2)

tprint (t2)


Output

1="dwarf"
2="elf"
3="human"

Now the t2 table has the names in alphabetic order, so we can just pull them out in sequence (with 'table.foreachi', or 'for ... ipairs') and use the value (eg. dwarf) to index into the main table.


t = { 

  dwarf = { str = 22, dex = 23, wis = 18 },
  human = { str = 20, dex = 20, wis = 20 },
  elf   = { str = 18, dex = 24, wis = 25 },

}

t2 = {}

table.foreach (t, function (k) table.insert (t2, k) end )

for k, v in ipairs (t2) do
  print ""
  print ("key = ", v)
  table.foreach (t [v], print)
end -- for

Output

key =  dwarf
str 22
dex 23
wis 18

key =  elf
str 18
dex 24
wis 25

key =  human
str 20
dex 20
wis 20

Converting a table into a string

For simple tables, table.concat can convert its values back into a string, like this:


t = 
  {
  "the", 
  "quick", 
  "brown", 
  "fox", 
  "jumped",
}

print (table.concat (t, ","))

Output

the,quick,brown,fox,jumped

You can also specify the range of the table to operate over. However this simple technique will not work in every case (for example, if the values had commas in them).

A more sophisticated version is the serialize function which is supplied in the exampscript.lua file which ships with MUSHclient. This handles nested tables and other considerations.

Nick Gammon Australia Forum Administrator Sun 30 Oct 2005 02:38 AM #2

Sophisticated table access through metatables

Tables in Lua can have an associated "metatable", where the metatable helps control access to the associated table. A couple of examples will show the general idea.

One thing you might want to do frequently is have default values for your table, so that if the value you want isn't in the base table it can be looked up elsewhere. The __index item (that is the word "index" with 2 underscores before it) can be placed in the metatable. It can be a function, in which case it can do whatever you want, or simply another "parent" table.

Another thing you might want to improve is how a table is printed if you do something like: print (mytable)

The default is something like "table: 00DE4610", however you might want to pluck a couple of fields out of the table and show those instead. The example below illustrates both of those techniques ...


races = { 

  dwarf = { str = 22, dex = 23, wis = 18 },
  human = { str = 20, dex = 20, wis = 20 },
  elf   = { str = 18, dex = 24, wis = 25, luck = 14 },

}

-- metatable that defines default values, and how to print the table

mt = { 

     -- default values
     __index = {str = 18, dex = 18, wis = 18, luck = 18 },


     -- how to print an item
     __tostring = function (race)
           return "str = " .. race.str .. 
                  ", dex = " .. race.dex
         end -- function
     }


-- set metatable for all tables in this table

for k, v in pairs (races) do setmetatable (v, mt) end

-- test it

print (races.dwarf.luck)  --> 18
print (races.human.luck)  --> 18
print (races.elf.luck)    --> 14

print (races.elf)  --> str = 18, dex = 24

Here we see that the dwarf and human "luck" figure is the default of 18 taken from the metatable, however the elf luck figure is from the original table.

Also when we printed races.elf the __tostring function was called which formatted the output as we desired.

Another interesting metatable item we can add is the __call one, which lets us pretend a table is a function, and "call" it. If we do that, the table in question is passed as the first argument, and any arguments to the call follow it.


function mt.__call (self, what)
    print ("function called, argument = " .. what)
    print ("luck = " .. self.luck)
  end -- function

races.elf ("hello there")

Output

function called, argument = hello there
luck = 14

This example appends to the previous one, so the metatable definition is still in force. Here I am using a slightly different syntax to declare a function in the table mt. It is semantically identical to:



mt.__call = function (self, what)

What happens when the races.elf function call is executed, is that the function mt.__call is called, passing the current table item (races.elf) as the first argument, so it can print its luck value.

Nick Gammon Australia Forum Administrator Mon 31 Oct 2005 04:26 AM #3

There is more information about Lua tables in an earlier post I did: Tables in Lua.

Nick Gammon Australia Forum Administrator Sun 03 Sep 2006 03:50 AM #4

There is another, more general, way of sorting tables into key order. This is presented in the Programming in Lua book, 2nd edition.

What this does is present a general iterator function (pairsByKeys), that can take any table, and be used to iterate through it in key order.

What it does is use the general idea presented earlier in this thread, namely to copy the keys into a temporary table, sort the temporary table, and then process the temporary table in sequence.

However the temporary table, and the sequencing, is stored as upvalues for the iterator function.


function pairsByKeys (t, f)
  local a = {}
  -- build temporary table of the keys
  for n in pairs (t) do 
    table.insert (a, n) 
  end
  table.sort (a, f)  -- sort using supplied function, if any
  local i = 0        -- iterator variable
  return function () -- iterator function
    i = i + 1
    return a[i], t[a[i]]
  end  -- iterator function
end -- pairsByKeys
  
t = { 

  dwarf = { str = 22, dex = 23, wis = 18 },
  human = { str = 20, dex = 20, wis = 20 },
  elf   = { str = 18, dex = 24, wis = 25 },

}

for k, v in pairsByKeys (t) do
  print (k, v)
end

Output

dwarf	table: 00305A40
elf	table: 00306E20
human	table: 00305A10

This function ships with MUSHclient these days as a separate file, so the above example could be written as:


require "pairsbykeys"

t = { 

  dwarf = { str = 22, dex = 23, wis = 18 },
  human = { str = 20, dex = 20, wis = 20 },
  elf   = { str = 18, dex = 24, wis = 25 },

}

for k, v in pairsByKeys (t) do
  print (k, v)
end

If you want to sort in a different sequence to the default "less than", you can supply your own function:


function gt (a, b)
  return a > b
end -- gt

for k, v in pairsByKeys (t, gt) do
  print (k, v)
end

Output

human	table: 00305A10
elf	table: 00306E20
dwarf	table: 00305A40

Nick Gammon Australia Forum Administrator Sat 30 Sep 2006 10:50 PM #5

Tables in Lua 5.1

There are a couple of modifications in Lua 5.1 to the table design.

You can no longer "set the length" of a table with table.setn.
You can get the length of the "vector part" of the table with the # operator. eg.
```
t = { "all", "good", "boys" }
print (#t) --> 3
```

Deleting all from a table

To delete all items from a table, you can do two things:

Simply make a new table. eg.


t = { "all", "good", "boys" }
print (#t) --> 3
t = {}  -- replace with empty table
print (#t) --> 0

This technique is the easiest, however will not work if you have references to that table elsewhere. For example:


t1 = { "all", "good", "boys" }
t2 = t1
print (#t1) --> 3
t1 = {}  -- replace with empty table
print (#t1) --> 0 (t1 is a new, empty table)
print (#t2) --> 3 (t2 is still the original table)

Delete all items. eg.
```
t = { "all", "good", "boys" }
print (#t) --> 3
for k in pairs (t) do
  t [k] = nil
end -- for
print (#t) --> 0
```
This actually removes items from the existing table, so it will also remove from any references as well.

Table efficiency

Earlier I said that table keys are hashed for fast lookup. This is not strictly true, as I discovered recently.

Lua actually stores tables internally in two parts:

A "vector" part, which is for numerically keyed items, starting at 1, and with few gaps in the sequence.
A "hash" part for all other keys, including large gaps in the numeric sequence.

Thus, a table with keys of (say) 1 to 100, will be stored as a vector internally, and thus have fast lookup, as it doesn't actually store (or hash) the keys, but directly indexes into the vector part.

However if you leave a large gap, like this:


t = {}
t [1] = "Nick"
t [10000] = "Gammon"

In this case the key of 1 will be in the vector part, and the key of 10000 will be in the hash part. Otherwise it would leave 9999 empty slots in the vector, which would waste a lot of space.

Because of this, you can safely use Lua tables in situations where you really want a vector (like a vector in STL), without worrying that it is slow because of always doing hashed lookups.

Nick Gammon Australia Forum Administrator Thu 22 May 2008 10:14 PM #6

How to see if a table is empty

If you want to know if a table is empty, or not, you can test:


if next (t) == nil then
  -- table t is empty
end -- if empty

The "next" function is a core Lua function that returns the next item in a table, given an existing key, or the first item if the key is nil. Thus, if supplied an initial key of nil, and it returns nil, then there must be no items in the table.

For more details about the "next" function, see:

http://www.gammon.com.au/scripts/doc.php?lua=next

The first argument to "next" is the table, the second argument is the key we want to move on from. With Lua, if you don't supply an argument (as in the case above) it defaults to nil.

Mahony Sun 24 Aug 2014 09:13 PM #7

Hi
Great post! But I miss only one thing. The table is always defined directly. Like t = {"one two", "three four"} at the beginning of the script. I need to define it from a variable:
variable "num" contains "one two", "three four"

Now I try
t = {GetVariable("num")}

I try to print the table t
for k, v in pairs (t) do
print (v)
end

and I get
"one two", "three four"

not

one two
three four

that I would expect. Where do I do misstake?
Thank you

Nick Gammon Australia Forum Administrator Sun 24 Aug 2014 10:29 PM #8

http://www.gammon.com.au/scripts/doc.php?lua=utils.split


num = "one two, three four"

t = utils.split (num, ",")

for k, v in pairs (t) do
  print (v)
end

Nick Gammon Australia Forum Administrator Sun 24 Aug 2014 10:30 PM #9

Also see about serializing:

http://www.gammon.com.au/forum/?id=4960

Nick Gammon Australia Forum Administrator Sun 24 Aug 2014 10:30 PM #10

I'm locking this thread because it is old. If you want to ask more questions please start a new one.