This forum is a read-only archive of the Gammon Software forum (2000–2026). No new posts can be made. Search the archive.

How to process incoming serial data without blocking

Posted by Nick Gammon on Sat 12 Nov 2011 08:53 PM — 4 posts, 213,018 views.

Nick Gammon Australia Forum Administrator #0
This page can be quickly reached from the link: http://www.gammon.com.au/serial


Something that seems to "throw" beginners to programming on a microprocessor is how to receive incoming serial data without "blocking". That is, being able to do other things while the text is arriving.

For a general discussion about the async serial protocol see:

http://www.gammon.com.au/forum/?id=10894


Serial data comes in fairly slowly (depending on the baud rate). For example at 9600 baud you get 960 characters per second, which means each one takes 1/960 of a second, that is 1.041 mS each. Whilst a millisecond might not sound like much, once you are expecting 80 characters it starts to add up.

Buffering input


The code below illustrates how, inside the main "loop" function, if any incoming data is received it is placed inside a buffer. Once a terminating character (typically a newline) is received the buffer is sent off to another function for processing, and the buffer reset to be empty.


/*
Example of processing incoming serial data without blocking.

Author:   Nick Gammon
Date:     13 November 2011. 
Modified: 31 August 2013.

Released for public use.
*/

// how much serial data we expect before a newline
const unsigned int MAX_INPUT = 50;

void setup ()
  {
  Serial.begin (115200);
  } // end of setup

// here to process incoming serial data after a terminator received
void process_data (const char * data)
  {
  // for now just display it
  // (but you could compare it to some value, convert to an integer, etc.)
  Serial.println (data);
  }  // end of process_data
  
void processIncomingByte (const byte inByte)
  {
  static char input_line [MAX_INPUT];
  static unsigned int input_pos = 0;

  switch (inByte)
    {

    case '\n':   // end of text
      input_line [input_pos] = 0;  // terminating null byte
      
      // terminator reached! process input_line here ...
      process_data (input_line);
      
      // reset buffer for next time
      input_pos = 0;  
      break;

    case '\r':   // discard carriage return
      break;

    default:
      // keep adding if not full ... allow for terminating null byte
      if (input_pos < (MAX_INPUT - 1))
        input_line [input_pos++] = inByte;
      break;

    }  // end of switch
   
  } // end of processIncomingByte  

void loop()
  {
  // if serial data available, process it
  while (Serial.available () > 0)
    processIncomingByte (Serial.read ());
    
  // do other stuff here like testing digital input (button presses) ...

  }  // end of loop


[EDIT] Modified 31 August 2013 to move the processing of each byte into a separate function for clarity.

[EDIT] Modified 15 November 2014 to change "if (Serial.available () > 0)" to "while (Serial.available () > 0)" which allows for lengthier processing inside the "other stuff" in the main loop.
Amended on Fri 14 Nov 2014 08:44 PM by Nick Gammon
Nick Gammon Australia Forum Administrator #1
State Machine


Another way of processing incoming data, without blocking, is to set up a "state machine". Effectively this means looking at each byte in the input stream, and handling it depending on the current state.

As an example, say you had this coming into the serial port:


R4500S80G3


Where Rnnn is RPM, Snnnn is speed, and Gnnnn is the gear setting.

The state machine below switches state when it gets a letter "R", "S" or "G". Otherwise it processes incoming digits by multiplying the previous result by 10, and adding in the new one.

When switching states, if first handles the previous state. So for example, after getting R4500 when the "S" arrives, we call the ProcessRPM function, passing it 4500.

This has the advantage of handling long messages without even needing any buffer, thus saving RAM. You can also process message as soon as the state changes, rather than waiting for end-of-line.

Example state-machine code


// Example state machine reading serial input
// Author: Nick Gammon
// Date: 17 December 2011

// the possible states of the state-machine
typedef enum {  NONE, GOT_R, GOT_S, GOT_G } states;

// current state-machine state
states state = NONE;
// current partial number
unsigned int currentValue;

void setup ()
{
  Serial.begin (115200);
  state = NONE;
}  // end of setup

void processRPM (const unsigned int value)
{
  // do something with RPM 
  Serial.print ("RPM = ");
  Serial.println (value);
} // end of processRPM

void processSpeed (const unsigned int value)
{
  // do something with speed 
  Serial.print ("Speed = ");
  Serial.println (value);
} // end of processSpeed

void processGear (const unsigned int value)
{
  // do something with gear 
  Serial.print ("Gear = ");
  Serial.println (value);  
} // end of processGear

void handlePreviousState ()
{
  switch (state)
  {
  case GOT_R:
    processRPM (currentValue);
    break;
  case GOT_S:
    processSpeed (currentValue);
    break;
  case GOT_G:
    processGear (currentValue);
    break;
  }  // end of switch  

  currentValue = 0; 
}  // end of handlePreviousState

void processIncomingByte (const byte c)
{
  if (isdigit (c))
  {
    currentValue *= 10;
    currentValue += c - '0';
  }  // end of digit
  else 
  {

    // The end of the number signals a state change
    handlePreviousState ();

    // set the new state, if we recognize it
    switch (c)
    {
    case 'R':
      state = GOT_R;
      break;
    case 'S':
      state = GOT_S;
      break;
    case 'G':
      state = GOT_G;
      break;
    default:
      state = NONE;
      break;
    }  // end of switch on incoming byte
  } // end of not digit  
  
} // end of processIncomingByte

void loop ()
{
  while (Serial.available ())
    processIncomingByte (Serial.read ());

  // do other stuff in loop as required
  
}  // end of loop


You could change "unsigned int" to "unsigned long" if you were expecting numbers larger than 65535.
Amended on Fri 14 Nov 2014 08:43 PM by Nick Gammon
Nick Gammon Australia Forum Administrator #2
Why do I need to see if data is available?


This question comes up most days on the Arduino forum, so I'll demonstrate why, as a picture is worth a thousand words.

Typical beginner code will look like this:


  byte a, b, c, d, e;
  if (Serial.available () > 0)
    {
    a = Serial.read ();  
    b = Serial.read ();  
    c = Serial.read ();  
    d = Serial.read ();  
    e = Serial.read ();  
    }  // end of if serial data available


The intention, clearly, is to wait for some serial data, and then read it into a, b, c, d and e.

But why doesn't this work? Because the processor is much faster than the serial data. This example program demonstrates:


const byte LED = 13;

void flash ()
  {
  digitalWrite (LED, HIGH);
  digitalWrite (LED, LOW);
  }  // end of flash

void setup ()
 {
 pinMode (LED, OUTPUT);
 Serial.begin (115200);
 }  // end of setup
 
void loop ()
  {
  byte a, b, c, d, e;
  if (Serial.available () > 0)
    {
    flash ();
    a = Serial.read ();  
    flash ();
    b = Serial.read ();  
    flash ();
    c = Serial.read ();  
    flash ();
    d = Serial.read ();  
    flash ();
    e = Serial.read ();  
    flash ();  
    
    exit (0);  // stop program to save confusion
    }  // end of if serial data available
  } // end of loop


If added a "flash" (brief toggle of the LED on pin 13) before each serial read. By looking at that with the logic analyzer we can see how quickly the program reads (and bear in mind the flashing code will have slowed it down a bit) ...



The graphic shows that whilst "a" is read OK (after all, it was after the Serial.available) the other variables b, c, d and e are all read even before the next letter has fully arrived!

And that was at the fastest baud rate. At slower baud rates (eg. 9600) the effect is much more pronounced:


Amended on Sat 28 Jan 2012 08:15 PM by Nick Gammon
Nick Gammon Australia Forum Administrator #3
How to send and receive numbers


To send a number (greater than 0 to 9) from one Arduino to another reliably you need to do a couple of things.

  • Send some sort of "start number" delimiter (in this example, the "<" character).
  • "Print" the number, which turns it into a string of ASCII characters.
  • Send some sort of "end number" delimiter (in this example, the ">" character).


Example sketch, sending 10 random numbers out the serial port:


// Example of sending numbers by Serial
// Author: Nick Gammon
// Date: 31 March 2012

const char startOfNumberDelimiter = '<';
const char endOfNumberDelimiter   = '>';

void setup ()
  { 
  srand (42);
  Serial.begin (115200);
  } // end of setup
  
void loop ()
  {
  Serial.println ("Starting ...");
  for (int i = 0; i < 10; i++)
    {
    Serial.print (startOfNumberDelimiter);    
    Serial.print (rand ());    // send the number
    Serial.print (endOfNumberDelimiter);  
    Serial.println ();
    }  // end of for

  delay (5000);
  }  // end of loop


Output:

Starting ...
<17766>
<11151>
<23481>
<32503>
<7018>
<25817>
<28529>
<9160>
<16666>
<13513>




To receive the numbers we need to detect in our main loop whether or not anything has arrived in the serial port. If so, we can check for the special "start" and "end" characters. These tell us whether to start a new number, and when it has finished arriving.

// Example of receiving numbers by Serial
// Author: Nick Gammon
// Date: 31 March 2012

const char startOfNumberDelimiter = '<';
const char endOfNumberDelimiter   = '>';

void setup ()
  { 
  Serial.begin (115200);
  Serial.println ("Starting ...");
  } // end of setup
  
void processNumber (const long n)
  {
  Serial.println (n);
  }  // end of processNumber
  
void processInput ()
  {
  static long receivedNumber = 0;
  static boolean negative = false;
  
  byte c = Serial.read ();
  
  switch (c)
    {
      
    case endOfNumberDelimiter:  
      if (negative) 
        processNumber (- receivedNumber); 
      else
        processNumber (receivedNumber); 

    // fall through to start a new number
    case startOfNumberDelimiter: 
      receivedNumber = 0; 
      negative = false;
      break;
      
    case '0' ... '9': 
      receivedNumber *= 10;
      receivedNumber += c - '0';
      break;
      
    case '-':
      negative = true;
      break;
      
    } // end of switch  
  }  // end of processInput
  
void loop ()
  {
  
  while (Serial.available ())
    processInput ();
    
  // do other stuff here
  } // end of loop


Output:

Starting ...
17766
11151
23481
32503
7018
25817
28529
9160
16666
13513


You will note that the receiving sketch does not have any delay calls in it. It runs at full speed, detecting incoming numbers as fast as it can. When a complete number has arrived it is processed (in processNumber).

You can choose different delimiters, just make sure that the sending and the receiving end use the same ones. The receiving sketch is designed so that you can omit the starting delimiter between numbers, as an ending delimiter is assumed to start a new number.

[EDIT] As pointed out on the Arduino forum, the code above doesn't check that the minus sign is at the start of the number. You may want to add a check for that if it is possible that invalid data would arrive.

Also, for robustness, you may want to add extra checks (like CRC checks) to detect dropped or substituted data.

The code above illustrates how to receive serial data and break it up into discrete numbers. It doesn't have any error-checking in it.