Gammon Forum : Electronics : Microprocessors : How to process incoming serial data without blocking

Log on

Gammon Forum

Entire forum

Electronics

Microprocessors

How to process incoming serial data without blocking

How to process incoming serial data without blocking

Postings by administrators only.

Refresh page

Posted by Nick Gammon Australia (22,928 posts) bio Forum Administrator

Date

Sat 12 Nov 2011 08:53 PM (UTC)

Amended on Fri 14 Nov 2014 08:44 PM (UTC) by Nick Gammon

Message

This page can be quickly reached from the link: http://www.gammon.com.au/serial

Something that seems to "throw" beginners to programming on a microprocessor is how to receive incoming serial data without "blocking". That is, being able to do other things while the text is arriving.

For a general discussion about the async serial protocol see:

http://www.gammon.com.au/forum/?id=10894

Serial data comes in fairly slowly (depending on the baud rate). For example at 9600 baud you get 960 characters per second, which means each one takes 1/960 of a second, that is 1.041 mS each. Whilst a millisecond might not sound like much, once you are expecting 80 characters it starts to add up.

Buffering input

The code below illustrates how, inside the main "loop" function, if any incoming data is received it is placed inside a buffer. Once a terminating character (typically a newline) is received the buffer is sent off to another function for processing, and the buffer reset to be empty.


/*
Example of processing incoming serial data without blocking.

Author:   Nick Gammon
Date:     13 November 2011. 
Modified: 31 August 2013.

Released for public use.
*/

// how much serial data we expect before a newline
const unsigned int MAX_INPUT = 50;

void setup ()
  {
  Serial.begin (115200);
  } // end of setup

// here to process incoming serial data after a terminator received
void process_data (const char * data)
  {
  // for now just display it
  // (but you could compare it to some value, convert to an integer, etc.)
  Serial.println (data);
  }  // end of process_data
  
void processIncomingByte (const byte inByte)
  {
  static char input_line [MAX_INPUT];
  static unsigned int input_pos = 0;

  switch (inByte)
    {

    case '\n':   // end of text
      input_line [input_pos] = 0;  // terminating null byte
      
      // terminator reached! process input_line here ...
      process_data (input_line);
      
      // reset buffer for next time
      input_pos = 0;  
      break;

    case '\r':   // discard carriage return
      break;

    default:
      // keep adding if not full ... allow for terminating null byte
      if (input_pos < (MAX_INPUT - 1))
        input_line [input_pos++] = inByte;
      break;

    }  // end of switch
   
  } // end of processIncomingByte  

void loop()
  {
  // if serial data available, process it
  while (Serial.available () > 0)
    processIncomingByte (Serial.read ());
    
  // do other stuff here like testing digital input (button presses) ...

  }  // end of loop

[EDIT] Modified 31 August 2013 to move the processing of each byte into a separate function for clarity.

[EDIT] Modified 15 November 2014 to change "if (Serial.available () > 0)" to "while (Serial.available () > 0)" which allows for lengthier processing inside the "other stuff" in the main loop.

- Nick Gammon

www.gammon.com.au, www.mushclient.com

top

Posted by Nick Gammon Australia (22,928 posts) bio Forum Administrator

Date

Reply #1 on Fri 16 Dec 2011 11:27 PM (UTC)

Amended on Fri 14 Nov 2014 08:43 PM (UTC) by Nick Gammon

Message

State Machine

Another way of processing incoming data, without blocking, is to set up a "state machine". Effectively this means looking at each byte in the input stream, and handling it depending on the current state.

As an example, say you had this coming into the serial port:


R4500S80G3

Where Rnnn is RPM, Snnnn is speed, and Gnnnn is the gear setting.

The state machine below switches state when it gets a letter "R", "S" or "G". Otherwise it processes incoming digits by multiplying the previous result by 10, and adding in the new one.

When switching states, if first handles the previous state. So for example, after getting R4500 when the "S" arrives, we call the ProcessRPM function, passing it 4500.

This has the advantage of handling long messages without even needing any buffer, thus saving RAM. You can also process message as soon as the state changes, rather than waiting for end-of-line.

Example state-machine code


// Example state machine reading serial input
// Author: Nick Gammon
// Date: 17 December 2011

// the possible states of the state-machine
typedef enum {  NONE, GOT_R, GOT_S, GOT_G } states;

// current state-machine state
states state = NONE;
// current partial number
unsigned int currentValue;

void setup ()
{
  Serial.begin (115200);
  state = NONE;
}  // end of setup

void processRPM (const unsigned int value)
{
  // do something with RPM 
  Serial.print ("RPM = ");
  Serial.println (value);
} // end of processRPM

void processSpeed (const unsigned int value)
{
  // do something with speed 
  Serial.print ("Speed = ");
  Serial.println (value);
} // end of processSpeed

void processGear (const unsigned int value)
{
  // do something with gear 
  Serial.print ("Gear = ");
  Serial.println (value);  
} // end of processGear

void handlePreviousState ()
{
  switch (state)
  {
  case GOT_R:
    processRPM (currentValue);
    break;
  case GOT_S:
    processSpeed (currentValue);
    break;
  case GOT_G:
    processGear (currentValue);
    break;
  }  // end of switch  

  currentValue = 0; 
}  // end of handlePreviousState

void processIncomingByte (const byte c)
{
  if (isdigit (c))
  {
    currentValue *= 10;
    currentValue += c - '0';
  }  // end of digit
  else 
  {

    // The end of the number signals a state change
    handlePreviousState ();

    // set the new state, if we recognize it
    switch (c)
    {
    case 'R':
      state = GOT_R;
      break;
    case 'S':
      state = GOT_S;
      break;
    case 'G':
      state = GOT_G;
      break;
    default:
      state = NONE;
      break;
    }  // end of switch on incoming byte
  } // end of not digit  
  
} // end of processIncomingByte

void loop ()
{
  while (Serial.available ())
    processIncomingByte (Serial.read ());

  // do other stuff in loop as required
  
}  // end of loop

You could change "unsigned int" to "unsigned long" if you were expecting numbers larger than 65535.

- Nick Gammon

www.gammon.com.au, www.mushclient.com

top

Posted by Nick Gammon Australia (22,928 posts) bio Forum Administrator

Date

Reply #2 on Sat 28 Jan 2012 08:09 PM (UTC)

Amended on Sat 28 Jan 2012 08:15 PM (UTC) by Nick Gammon

Message

Why do I need to see if data is available?

This question comes up most days on the Arduino forum, so I'll demonstrate why, as a picture is worth a thousand words.

Typical beginner code will look like this:


  byte a, b, c, d, e;
  if (Serial.available () > 0)
    {
    a = Serial.read ();  
    b = Serial.read ();  
    c = Serial.read ();  
    d = Serial.read ();  
    e = Serial.read ();  
    }  // end of if serial data available

The intention, clearly, is to wait for some serial data, and then read it into a, b, c, d and e.

But why doesn't this work? Because the processor is much faster than the serial data. This example program demonstrates:


const byte LED = 13;

void flash ()
  {
  digitalWrite (LED, HIGH);
  digitalWrite (LED, LOW);
  }  // end of flash

void setup ()
 {
 pinMode (LED, OUTPUT);
 Serial.begin (115200);
 }  // end of setup
 
void loop ()
  {
  byte a, b, c, d, e;
  if (Serial.available () > 0)
    {
    flash ();
    a = Serial.read ();  
    flash ();
    b = Serial.read ();  
    flash ();
    c = Serial.read ();  
    flash ();
    d = Serial.read ();  
    flash ();
    e = Serial.read ();  
    flash ();  
    
    exit (0);  // stop program to save confusion
    }  // end of if serial data available
  } // end of loop

If added a "flash" (brief toggle of the LED on pin 13) before each serial read. By looking at that with the logic analyzer we can see how quickly the program reads (and bear in mind the flashing code will have slowed it down a bit) ...

The graphic shows that whilst "a" is read OK (after all, it was after the Serial.available) the other variables b, c, d and e are all read even before the next letter has fully arrived!

And that was at the fastest baud rate. At slower baud rates (eg. 9600) the effect is much more pronounced:

- Nick Gammon

www.gammon.com.au, www.mushclient.com

top

Posted by Nick Gammon Australia (22,928 posts) bio Forum Administrator

Date

Reply #3 on Sat 31 Mar 2012 02:54 AM (UTC)

Amended on Fri 14 Nov 2014 08:44 PM (UTC) by Nick Gammon

Message

How to send and receive numbers

To send a number (greater than 0 to 9) from one Arduino to another reliably you need to do a couple of things.

Send some sort of "start number" delimiter (in this example, the "<" character).
"Print" the number, which turns it into a string of ASCII characters.
Send some sort of "end number" delimiter (in this example, the ">" character).

Example sketch, sending 10 random numbers out the serial port:


// Example of sending numbers by Serial
// Author: Nick Gammon
// Date: 31 March 2012

const char startOfNumberDelimiter = '<';
const char endOfNumberDelimiter   = '>';

void setup ()
  { 
  srand (42);
  Serial.begin (115200);
  } // end of setup
  
void loop ()
  {
  Serial.println ("Starting ...");
  for (int i = 0; i < 10; i++)
    {
    Serial.print (startOfNumberDelimiter);    
    Serial.print (rand ());    // send the number
    Serial.print (endOfNumberDelimiter);  
    Serial.println ();
    }  // end of for

  delay (5000);
  }  // end of loop

Output:

Starting ...
<17766>
<11151>
<23481>
<32503>
<7018>
<25817>
<28529>
<9160>
<16666>
<13513>

To receive the numbers we need to detect in our main loop whether or not anything has arrived in the serial port. If so, we can check for the special "start" and "end" characters. These tell us whether to start a new number, and when it has finished arriving.

// Example of receiving numbers by Serial
// Author: Nick Gammon
// Date: 31 March 2012

const char startOfNumberDelimiter = '<';
const char endOfNumberDelimiter   = '>';

void setup ()
  { 
  Serial.begin (115200);
  Serial.println ("Starting ...");
  } // end of setup
  
void processNumber (const long n)
  {
  Serial.println (n);
  }  // end of processNumber
  
void processInput ()
  {
  static long receivedNumber = 0;
  static boolean negative = false;
  
  byte c = Serial.read ();
  
  switch (c)
    {
      
    case endOfNumberDelimiter:  
      if (negative) 
        processNumber (- receivedNumber); 
      else
        processNumber (receivedNumber); 

    // fall through to start a new number
    case startOfNumberDelimiter: 
      receivedNumber = 0; 
      negative = false;
      break;
      
    case '0' ... '9': 
      receivedNumber *= 10;
      receivedNumber += c - '0';
      break;
      
    case '-':
      negative = true;
      break;
      
    } // end of switch  
  }  // end of processInput
  
void loop ()
  {
  
  while (Serial.available ())
    processInput ();
    
  // do other stuff here
  } // end of loop

Output:

Starting ...
17766
11151
23481
32503
7018
25817
28529
9160
16666
13513

You will note that the receiving sketch does not have any delay calls in it. It runs at full speed, detecting incoming numbers as fast as it can. When a complete number has arrived it is processed (in processNumber).

You can choose different delimiters, just make sure that the sending and the receiving end use the same ones. The receiving sketch is designed so that you can omit the starting delimiter between numbers, as an ending delimiter is assumed to start a new number.

[EDIT] As pointed out on the Arduino forum, the code above doesn't check that the minus sign is at the start of the number. You may want to add a check for that if it is possible that invalid data would arrive.

Also, for robustness, you may want to add extra checks (like CRC checks) to detect dropped or substituted data.

The code above illustrates how to receive serial data and break it up into discrete numbers. It doesn't have any error-checking in it.

- Nick Gammon

www.gammon.com.au, www.mushclient.com

top

The dates and times for posts above are shown in Universal Co-ordinated Time (UTC).

To show them in your local time you can join the forum, and then set the 'time correction' field in your profile to the number of hours difference between your location and UTC time.

187,667 views.

Postings by administrators only.

Refresh page

Go to topic: Search the forum

top

Quick links: MUSHclient. MUSHclient help. Forum shortcuts. Posting templates. Lua modules. Lua documentation.

Information and images on this site are licensed under the Creative Commons Attribution 3.0 Australia License unless stated otherwise.