Newer, Better, Cooler Bitmask-Flags

Posted by David Haley on Fri 28 May 2004 11:58 PM — 24 posts, 96,523 views.

USA #0
For all those fortunate enough to be using C++, I suggest you check out the bitset class.

http://www.josuttis.com/libbook/cont/bitset1.cpp.html

Basically, you have a class that can be expanded to as many flags as you want. I suspect that the template figures out how many bytes it needs to allocate, and then when you ask it for a given entry it automagically figures out which byte to mask against.

This is a pristine solution to the problem of expandable bitmask flags. I pity the poor souls stuck with C. :-)
USA #1
Nice, very nice. Looks like I'll be seeing about getting this used in my codebase. Was just wondering if you've run across a link with better information. This one doesn't seem to give a complete example of how to use it. Went looking for something better but I'm not really finding much.
USA #2
Yes, I'll definitely be using this gem as well. This is what I love about the STL: it has so much stuff done for you already that really lets you worry about more important things, and not pesky little flags and stuff. :-)

The link I gave you is from Josuttis's excellent book. Unfortunately, he only made his examples publicly available. :) The best starting point I would suggest is:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vcstdlib/html/vclrfBitsetbitset.asp
which is the MSDN page for the bitset.

That being said the example seems to cover most of it. Coupled with the MSDN example, I think we can pretty much figure it out. It seems that if you write it to an ostream, it gives the binary representation. I'd imagine that reading from an istream into a bitset produces much the same results.

Here is the index page will all his examples:
http://www.josuttis.com/libbook/idx.html
USA #3
Cool, checked the MSDN link and it has all the stuff you need to make use of this. Excellent find. I went in and changed one of the small sections of code that used the usual BV00 type definitions. It converted over flawlessly. Not only that, but using the to_ulong() conversion is perfect for saving numerical flag values. The file format doesn't need to change, and using an fread_number to load the value back into the bitset worked perfectly.

I'll expand my use of it to some more isolated areas of the code and see what I get from it. If things work out with that, converting all of the flag types to use this should be easy. I'm so glad I chose to save most of my flag values as text strings now :)
USA #4
Does to_ulong work even if you have more than 32 entries in your bitset? I haven't played around with this a lot yet so I wouldn't know.

And yes, I too have discovered the wisdom of saving as much as possible as text and not as evil numbers. :)
USA #5
I doubt to_ulong() will do the job for a conversion if you're above BV32. So for stuff in the current "extended" bitvector style, using text to save the fields is probably still best. None of the information I've found so far has provided any examples of saving bitset stuff to files, so I'm winging it in this area as it is.
USA #6

char *bitset_string( unsigned long flag, int flagmax, char *const flagarray[] )
{
   static char buf[MSL];
   int x;

   buf[0] = '\0';
   for( x = 0; x < flagmax; x++ )
	if( IS_SET( flag, 1 << x ) )
	{
	   mudstrlcat( buf, flagarray[x], MSL );
         /* don't catenate a blank if the last char is blank  --Gorog */
         if( buf[0] != '\0' && ' ' != buf[strlen(buf)-1] )
            mudstrlcat( buf, " ", MSL );
	}

   if( ( x = strlen( buf ) ) > 0 )
      buf[--x] = '\0';

   return buf;
}


The above code is what I cooked up to do conversions on the new bitset stuff from the bits that are set to the string which can represent them for saving in files and such. It works great, provided you don't go over what an unsigned long can hold. Trouble is, an unsigned long is insufficient for any of my extended bitvector cases. Further, I've apparently discovered that the to_string() conversion for std::bitset isn't available in g++ 3.3x which is the compiler I'm using.

So this creates a dillema. How do I manage to convert my extended cases into std::bitset and still be able to manipulate them?

As a quick example, if I try to print one of the normal ones to string format, this is the error I get:

channels.c: In function `void do_showchannels(char_data*, char*)':
channels.c:429: error: no matching function for call to `std::bitset<2u>::to_string()'

Should be noted also that this error triggers itself on g++ 3.4.0 as well, so if this is something the GNU C++ compiler is supposed to know about, perhaps I've found a bug that needs to be brought to GNU's attention?
Amended on Sat 26 Jun 2004 11:55 PM by Samson
Australia Forum Administrator #7
According to Josuttis book p 468:

Quote:


The function (to_string) is a template function that is parameterized only by return type. According to the language rules, you must write the following:

bitset<50> b;
b.template to_string<char, char_traits<char>,allocator<char> > ()


What this means a bit more plainly is that you cannot just use to_string, you must use the template code. The example program below works under g++ 3.2.2.

#include <bitset>
#include <iostream>

using namespace std;

int main (void)
  {
  bitset<50> mybits;

  mybits.set (1);
  mybits.set (5);
  mybits.set (40);

  cout << "bits = " <<
    mybits.template to_string<char, char_traits<char>,allocator<char> > () <<
    endl;

  }




Output


bits = 00000000010000000000000000000000000000000000100010


Hope this helps.
Amended on Sun 27 Jun 2004 08:01 AM by Nick Gammon
Australia Forum Administrator #8
You could make a define to do the dirty work for you, like this:


#include <bitset>
#include <iostream>

using namespace std;

#define bitset_to_string(arg) \
  arg.template to_string<char, char_traits<char>,allocator<char> > ()

int main (void)
  {
  bitset<50> mybits;

  mybits.set (1);
  mybits.set (5);
  mybits.set (40);

  cout << "bits = " << bitset_to_string (mybits) << endl;

  }

Australia Forum Administrator #9
Another, simpler, approach is to just use the output stream. This works and gives the same results:


#include <bitset>
#include <iostream>

using namespace std;

int main (void)
  {
  bitset<50> mybits;

  mybits.set (1);
  mybits.set (5);
  mybits.set (40);

  cout << "bits = " << mybits << endl;

  }


Australia Forum Administrator #10
Another approach again, if you want a string and not to go direct to cout, is to use a MAKE_STRING macro, like this:


#include <bitset>
#include <string>
#include <sstream>
#include <iostream>

using namespace std;

// this define lets you create a string on-the-fly
// you can use output modifiers in the string, eg.
// string mystr = MAKE_STRING (
//    "hello " << boolalpha << true << hex << 1234 << octal << 3456 << dec
//    << 123456 << scientific << 123.45 << fixed << 1234.567 " );

#define MAKE_STRING(msg) \
   (((ostringstream&) (ostringstream() << boolalpha << msg)).str())


int main (void)
  {
  bitset<50> mybits;

  mybits.set (1);
  mybits.set (5);
  mybits.set (40);

string s;

  s = MAKE_STRING (mybits);

  cout << "bits = " << s << endl;

  }
Amended on Sun 27 Jun 2004 11:36 PM by Nick Gammon
Australia Forum Administrator #11
The example below illustrates reading the bitset back in. I certainly wouldn't use a long to handle the bits, that defeats the whole purpose of being able to use more than 32 of them.


#include <bitset>
#include <string>
#include <sstream>
#include <iostream>

using namespace std;

#define MAKE_STRING(msg) \
   (((ostringstream&) (ostringstream() << boolalpha << msg)).str())

int main (void)
  {
  bitset<50> mybits;

  mybits.set (1);
  mybits.set (5);
  mybits.set (40);

string s;

  s = MAKE_STRING (mybits);

  cout << "bits = " << s << endl;

// now read it back in

  bitset<50> newbits;

  istringstream str (s);
  str >> newbits;
  if (str.fail ())
    {
    cout << "Invalid bits" << endl;
    return 1;
    }

  cout << "new  = " << newbits << endl;
  }
Amended on Sun 27 Jun 2004 11:36 PM by Nick Gammon
Australia Forum Administrator #12
Another approach, rather than using bitset, which limits you to deciding how many bits you want, in advance, is to use a map of bools (or a vector of bools for that matter). Here is one way:


#include <map>
#include <string>
#include <iostream>

using namespace std;

int main (void)
  {
  map <int, bool> flags;

  flags [1] = true;
  flags [5] = true;
  flags [40] = true;

  for (map <int, bool>::const_iterator i = flags.begin ();
       i != flags.end ();
       i++)
    cout << "bit " << i->first << " = " << i->second << endl;
  }



Output

bit 1 = 1
bit 5 = 1
bit 40 = 1


This system lets you use *any* flag numbers (eg. 10000) because the map simply stores the keys it is given.
Amended on Sun 27 Jun 2004 10:14 PM by Nick Gammon
Australia Forum Administrator #13
Once you have got that far, you can use named flags, which make it easier to program in the first place, and easier to read the area files, like this:


#include <map>
#include <string>
#include <iostream>

using namespace std;

int main (void)
  {
  map <string, bool> flags;

  flags ["dark"] = true;
  flags ["inside"] = true;
  flags ["nomob"] = true;

  for (map <string, bool>::const_iterator i = flags.begin ();
       i != flags.end ();
       i++)
    cout << "bit " << i->first << " = " << i->second << endl;
  }



Output

bit dark = 1
bit inside = 1
bit nomob = 1

USA #14
Well, all very good suggestions. After prodding a few things, I came up with this:


char *bitset_string( std::basic_string< char, std::char_traits<char>, std::allocator<char> > flags, int flagmax, char *const flagarray[] )
{
   static char buf[MSL];
   char newflags[MSL];
   int x;

   mudstrlcpy( newflags, flags.c_str(), MSL );

   invert( newflags, log_buf );
   mudstrlcpy( newflags, log_buf, MSL );

   buf[0] = '\0';
   for( x = 0; x < flagmax; x++ )
	if( newflags[x] == '1' )
	{
	   mudstrlcat( buf, flagarray[x], MSL );
         /* don't catenate a blank if the last char is blank  --Gorog */
         if( buf[0] != '\0' && ' ' != buf[strlen(buf)-1] )
            mudstrlcat( buf, " ", MSL );
	}

   if( ( x = strlen( buf ) ) > 0 )
      buf[--x] = '\0';

   return buf;
}


That's in build.c, right next to ext_flag_string and such. The first argument uses this macro:


#define BITSTRING( arg ) (arg).to_string< char, std::char_traits<char>, std::allocator<char> > ()


So basically now all I have to do is call it like so:

bitset_string( BITSTRING( channel->flags ), CHAN_MAXFLAG, chan_flags )

Yes, it's a bit of a hackish looking mess, but it's getting the job done and has already proven to work on a bitset with 65 members, which is far beyond what a ulong can hold.

The invert function was used because the normal bitset string is backward for this purpose. The lowest bits are at the rightmost portion of the string. The inversion made walking the string trivial at that point.
Amended on Sun 27 Jun 2004 10:47 PM by Samson
Australia Forum Administrator #15
Er, yes, it does look a bit of a mess. Why do you need all that complication? To reverse the bits, you can use the reverse algorithm on the string, like this:


#include <bitset>
#include <string>
#include <sstream>
#include <iostream>
#include <algorithm>

using namespace std;

// this define lets you create a string on-the-fly
// you can use output modifiers in the string, eg.
// string mystr = MAKE_STRING (
//    "hello " << boolalpha << true << hex << 1234 << octal << 3456 << dec
//    << 123456 << scientific << 123.45 << fixed << 1234.567 " );

#define MAKE_STRING(msg) \
   (((ostringstream&) (ostringstream() << boolalpha << msg)).str())


int main (void)
  {
  bitset<50> mybits;

  mybits.set (1);
  mybits.set (5);
  mybits.set (40);

string s;

  s = MAKE_STRING (mybits);

  cout << "bits = " << s << endl;
  reverse (s.begin (), s.end ());
  cout << "reversed bits = " << s << endl;

  }



Output

bits = 00000000010000000000000000000000000000000000100010
reversed bits = 01000100000000000000000000000000000000001000000000

USA #16

char *bitset_string( std::basic_string< char, std::char_traits<char>, std::allocator<char> > flags, int flagmax, char *const flagarray[] )
{
   static char buf[MSL];
   int x;

   reverse( flags.begin(), flags.end() );

   buf[0] = '\0';
   for( x = 0; x < flagmax; x++ )
	if( flags[x] == '1' )
	{
	   mudstrlcat( buf, flagarray[x], MSL );
         /* don't catenate a blank if the last char is blank  --Gorog */
         if( buf[0] != '\0' && ' ' != buf[strlen(buf)-1] )
            mudstrlcat( buf, " ", MSL );
	}

   if( ( x = strlen( buf ) ) > 0 )
      buf[--x] = '\0';

   return buf;
}


Mainly..... because I don't know that much about C++ yet. I'm delving into it bit by bit ( no pun intended! ) so every little thing helps. Didn't know about the reverse algorithm. Certainly made that particular portion of things much easier to work with though.
Australia Forum Administrator #17
OK, good. Now what does this routine do exactly? Turn a bitset into a string? Can't help thinking it can be replaced by one or two lines.
USA #18
Yes. It's purpose is to take a std::bitset and turn it into a string of names to represent the bits which are set. Much like flag_string does for the BVxx type, and ext_flag_string does for the EXT_BV types. Used for display in stats screens, identify spells, and file formats etc.

If it can be done in 2 lines I'd be rather impressed with that.
Australia Forum Administrator #19
Ah, I see. Converting a bitset to a string of names. Well, maybe 3 lines. :)

This still looks neater:


#ifdef WIN32
  #pragma warning( disable : 4786)
#endif

#include <bitset>
#include <vector>
#include <string>
#include <iostream>
#include <algorithm>

using namespace std;

char *  const   npc_race        [] =
{
"human", "elf", "dwarf", "halfling", "pixie", "vampire", "half-ogre",
"half-orc", "half-troll", "half-elf", "gith", "drow", "sea-elf",
"lizardman", "gnome", "r5", "r6", "r7", "r8", "troll",
"ant", "ape", "baboon", "bat", "bear", "bee",
"beetle", "boar", "bugbear", "cat", "dog", "dragon", "ferret", "fly",
"gargoyle", "gelatin", "ghoul", "gnoll", "gnome", "goblin", "golem",
"gorgon", "harpy", "hobgoblin", "kobold", "lizardman", "locust",
"lycanthrope", "minotaur", "mold", "mule", "neanderthal", "ooze", "orc",
"rat", "rustmonster", "shadow", "shapeshifter", "shrew", "shrieker",
"skeleton", "slime", "snake", "spider", "stirge", "thoul", "troglodyte",
"undead", "wight", "wolf", "worm", "zombie", "bovine", "canine", "feline",
"porcine", "mammal", "rodent", "avis", "reptile", "amphibian", "fish",
"crustacean", "insect", "spirit", "magical", "horse", "animal", "humanoid",
"monster", "god"
};


#define NUMITEMS(arg) (sizeof (arg) / sizeof (arg [0]))


template <class T>
string bitset_to_string (T bits, vector<string> names)
  {

  string s;

  for (size_t i = 0; i < bits.size (); i++)
    if (bits [i])
      s += names [i] + " ";
  

  return s;
  } // end of bitset_to_string

int main (void)
  {

  vector<string> npc_race_vector;
  bitset<NUMITEMS (npc_race)> race_flags;

  race_flags.set (5);
  race_flags.set (6);
  race_flags.set (70);

  copy (npc_race,
        &npc_race [ NUMITEMS (npc_race) ],
        back_inserter (npc_race_vector));

  cout << "result = " << bitset_to_string (race_flags, npc_race_vector) << endl;

  }



Output

result = vampire half-ogre worm




The important 3 lines are in bold.

What I have done here is use the NUMITEMS macro (to find how many flags are in the array), and convert the char * array into a vector (for simplicity, although this probably isn't necessary).

Then I used a templated function to save having to pass down the number of bits in the array.

Also by using:

bitset<NUMITEMS (npc_race)> race_flags;

I could let the bitset adjust itself to the number of flags actually needed, although there may be reasons to not do that.

Finally the bitset_to_string function scans the array of bits, and for each one set, copies the appropriate name into the string, appending a space.

The only problem is you will have a trailing space (unless no bits match) but you can always do a test and shorten the resulting string by one if necessary.

Amended on Mon 28 Jun 2004 10:53 PM by Nick Gammon
USA #20
Going by what you've posted above, I went on to modify my bitset_string function as follows:


template <class T>
char *bitset_string( T bits, char *const flagarray[] )
{
   static char buf[MSL];
   size_t x;

   buf[0] = '\0';
   for( x = 0; x < bits.size(); x++ )
   {
      if( bits.test(x) )
	{
	   mudstrlcat( buf, flagarray[x], MSL );
         /* don't catenate a blank if the last char is blank  --Gorog */
         if( buf[0] != '\0' && ' ' != buf[strlen(buf)-1] )
            mudstrlcat( buf, " ", MSL );
	}
   }

   if( ( x = strlen( buf ) ) > 0 )
      buf[--x] = '\0';

   return buf;
}


This has a prototype in mud.h as such:


template <class T>
char *bitset_string( T bits, char *const flagarray[] );


Called as such, for various different bitsets I want to turn into strings for the purpose of saving/displaying their contents:


	fprintf( fpout, "%s~\n", bitset_string( pMobIndex->actflags, act_flags ) );
      fprintf( fpout, "%s~\n", bitset_string( pMobIndex->affected_by, a_flags ) );


I was rather excited watching the results, it was compiling along just fine until it hit the linking stage where I got spammed silly with dozens of these:


o/act_info.o(.text+0x277c): In function `do_look':
/usr/include/c++/3.3.3/bitset:1064: undefined reference to `char* bitset_string<std::bitset<(unsigned)12> >(std::bitset<(unsigned)12>, char* const*)'
o/act_info.o(.text+0x27a4): In function `do_look':
/home/samson/Alsherok/src/act_info.c:1061: undefined reference to `char* bitset_string<std::bitset<(unsigned)42> >(std::bitset<(unsigned)42>, char* const*)'
o/act_wiz.o(.text+0x13e0): In function `do_rstat':
/home/samson/Alsherok/src/act_wiz.c:953: undefined reference to `char* bitset_string<std::bitset<(unsigned)42> >(std::bitset<(unsigned)42>, char* const*)'


Now, obviously I'm doing something wrong. I can't seriously expect that the compiler wants me to make 5 dozen different copies of bitset_string just in case. It seems to me all I'm missing is how to properly pass a std::bitset as an argument to a general function. There has to be away, but none of the web references I've found will tell me.
Australia Forum Administrator #21
I was caught by this a while ago, and the answer can be a bit obscure when you are not expecting it. :)

The way templates work is that the compiler generates the code required when it sees an instance of the templated code (not the template declaration).

Thus, you cannot prototype the function like you have done - the compiler is accepting the prototype, but is not aware to generate the *instance* required for each of the 5 cases.

The correct, and only, solution is to put the entire templated function into a header file, and include that in the places where it is wanted. This is how STL does it. If you have this:

#include <map>

(for example)

Then that has the entire code for generating maps inside the include file, and that has to be in the front of any module that wants to use it. And, there is no map.cpp file. It is all in the header.
USA #22
Ah, sweetness. That fixed it up nicely. No kidding on the not expecting it part. I sure didn't think it needed to be in a header file. But I'm still feeling my way through this for the most part. Thanks for the heads up :)
Australia Forum Administrator #23
Now that you are using STL and C++, it might be a good idea to look at moving away from static buffers for strings.

Depending on what MSL is defined as (4096 in my version) then you are either:

  • Using too much memory, and wasting (say) 4,000 bytes of RAM which is never used; or
  • Not allowing enough, in which case one day it will overflow and crash


The version below takes the same arguments as yours, and still returns a const char * string, but uses a string internally, which can never overflow. I have declared it as static so it doesn't go out of scope too soon ...



#ifdef WIN32
  #pragma warning( disable : 4786)
#endif

#include <bitset>
#include <string>
#include <iostream>

#include <stdio.h>

using namespace std;

char *  const   npc_race        [] =
{
"human", "elf", "dwarf", "halfling", "pixie", "vampire", "half-ogre",
"half-orc", "half-troll", "half-elf", "gith", "drow", "sea-elf",
"lizardman", "gnome", "r5", "r6", "r7", "r8", "troll",
"ant", "ape", "baboon", "bat", "bear", "bee",
"beetle", "boar", "bugbear", "cat", "dog", "dragon", "ferret", "fly",
"gargoyle", "gelatin", "ghoul", "gnoll", "gnome", "goblin", "golem",
"gorgon", "harpy", "hobgoblin", "kobold", "lizardman", "locust",
"lycanthrope", "minotaur", "mold", "mule", "neanderthal", "ooze", "orc",
"rat", "rustmonster", "shadow", "shapeshifter", "shrew", "shrieker",
"skeleton", "slime", "snake", "spider", "stirge", "thoul", "troglodyte",
"undead", "wight", "wolf", "worm", "zombie", "bovine", "canine", "feline",
"porcine", "mammal", "rodent", "avis", "reptile", "amphibian", "fish",
"crustacean", "insect", "spirit", "magical", "horse", "animal", "humanoid",
"monster", "god"
};


template <class T>
const char * bitset_to_string (T bits, char *const flagarray[] )
  {

  static string s;

  s.clear ();

  for (size_t i = 0; i < bits.size (); i++)
    if (bits [i])
      {
      if (!s.empty ())
        s += " ";  // separate multiple ones by a space
      s += flagarray [i];
      }

  return s.c_str ();
  } // end of bitset_to_string

int main (void)
  {

  bitset<100> race_flags;

  race_flags.set (5);
  race_flags.set (6);
  race_flags.set (70);

  printf ("results = '%s'\n", bitset_to_string (race_flags, npc_race) );

  }



The only problem with the code above is that it doesn't handle spaces inside your constants, which I think your code does. The version below which introduces a couple more routines that I wrote to trim spaces, will handle spaces at either end of the constants (see the spaces I added to "worm" below) ...


#ifdef WIN32
  #pragma warning( disable : 4786)
#endif

#include <bitset>
#include <string>
#include <iostream>

#include <stdio.h>

using namespace std;

char *  const   npc_race        [] =
{
"human", "elf", "dwarf", "halfling", "pixie", "vampire", "half-ogre",
"half-orc", "half-troll", "half-elf", "gith", "drow", "sea-elf",
"lizardman", "gnome", "r5", "r6", "r7", "r8", "troll",
"ant", "ape", "baboon", "bat", "bear", "bee",
"beetle", "boar", "bugbear", "cat", "dog", "dragon", "ferret", "fly",
"gargoyle", "gelatin", "ghoul", "gnoll", "gnome", "goblin", "golem",
"gorgon", "harpy", "hobgoblin", "kobold", "lizardman", "locust",
"lycanthrope", "minotaur", "mold", "mule", "neanderthal", "ooze", "orc",
"rat", "rustmonster", "shadow", "shapeshifter", "shrew", "shrieker",
"skeleton", "slime", "snake", "spider", "stirge", "thoul", "troglodyte",
"undead", "wight", "wolf", " worm ", "zombie", "bovine", "canine", "feline",
"porcine", "mammal", "rodent", "avis", "reptile", "amphibian", "fish",
"crustacean", "insect", "spirit", "magical", "horse", "animal", "humanoid",
"monster", "god"
};

#define SPACES " \t\r\n"  // default "spaces" for trimming strings

// trim spaces from right (you can define what spaces are)
string trim_right (const string & s, const string & t = SPACES);
// trim spaces from left
string trim_left (const string & s, const string & t = SPACES);
// trim spaces from both sides
string trim (const string & s, const string & t = SPACES);

inline string trim_right (const string & s, const string & t)
  {
  string d (s);
  string::size_type i (d.find_last_not_of (t));
  if (i == string::npos)
    return "";
  else
   return d.erase (d.find_last_not_of (t) + 1) ;
  }  // end of trim_right

inline string trim_left (const string & s, const string & t)
  {
  string d (s);
  return d.erase (0, s.find_first_not_of (t)) ;
  }  // end of trim_left

inline string trim (const string & s, const string & t)
  {
  string d (s);
  return trim_left (trim_right (d, t), t) ;
  }  // end of trim

template <class T>
const char * bitset_to_string (T bits, char *const flagarray[] )
  {

  static string s;

  s.clear ();

  for (size_t i = 0; i < bits.size (); i++)
    if (bits [i])
      s += trim (flagarray [i]) + " ";

  s = trim_right (s);  // get rid of final space

  return s.c_str ();
  } // end of bitset_to_string

int main (void)
  {

  bitset<100> race_flags;

  race_flags.set (5);
  race_flags.set (6);
  race_flags.set (70);

  printf ("results = '%s'\n", bitset_to_string (race_flags, npc_race) );

  }



Output

results = 'vampire half-ogre worm'




In the second example above, the implementations of trim, trim_left and trim_right *can* be moved into a cpp file (not a header file) because they are straight functions, not templated functions.