modelrail.otenko Trains, computers, vintage, retro, model railways… the lot…

More (Bit)Bang For Your Buck

Arduinos are a great hobbyist platform for projects, but you'll often encounter speed and memory issues when you try to squeeze in too many features. Fortunately, being an open source system, one can easily bypass the standard libraries and push these little units as hard as possible. Of course, once you're past the warranties and disclaimers, you're out on your own.

I've previously had to jump out of the IDE and use 'faster' code whilst using a Dreamcast Controller for my Model Railway, but that was a little more advanced than what we're about to do here. Today's goal is to bitbang the data stream to the Red LED Sign as quickly as possible. If you check that post, you'll see I'm already using digitalWriteFast, but it seems that we can go even quicker with the SPI library.

What's SPI, you ask? To steal from Arduino: Serial Peripheral Interface (SPI) is a synchronous serial data protocol used by microcontrollers for communicating with one or more peripheral devices quickly over short distances. As it turns out, that's exactly what we want to do! We want the display to be a slave and, well, we're just going to hack the right RX/TX lines together to get the data to flow. A fellow Arduino-er has done this here whilst interfacing with a 74HC595 and provides a few hints as to problems we might run in to.

Actually, now that I think of it, the data is to be sent out in bytes, but my screen is no where near the order of a clean multiplication of bytes... will have to work on that. Also... the data I read is the top 'line' of each letter, it's not a clean char in any memory array. Hmmm...

Test Patterns

Without pulling apart my currently-hacked-together-and-functional-display, I used another Uno I had lying around and hooked it up to the two spare LED panels. The whole display came with 5 panels, but I wasn't able to smoothly run them all at once, so these two were not in use.

The hook-up is pretty straight forward. The Wikipedia article on SPI gives a little more information on the bus and, more specifically, dictates which wires we need to use. We'll need the data output wire and the clock, known as MOSI, or Master Out-Slave In, and SCLK, the serial clock. Note that some devices read the data pin when the clock transitions from LOW to HIGH, and some vice-versa. Fortunately, this can all be configured via the SPI library.

With the following code, I managed to get a test pattern on the display...

#include <SPI.h>
void setup() {
  for (int p = 0; p <= 12; p++) {
    pinMode(p, OUTPUT);
    digitalWrite(p, LOW);
  }
  SPI.begin();
  SPI.setDataMode(SPI_MODE2);
  SPI.setBitOrder(LSBFIRST);
  SPI.setClockDivider(SPI_CLOCK_DIV128);
}

bool isEvenRow = false;
void loop() {
  isEvenRow = false;
  for (int row = 0; row < 7; row++) 
  {
    if (isEvenRow) {
      SPI.transfer(B01010101);
      SPI.transfer(B01010101);
      SPI.transfer(B01010101);
      SPI.transfer(B01010101);
    } else {
      SPI.transfer(B10101010);
      SPI.transfer(B10101010);
      SPI.transfer(B10101010);
      SPI.transfer(B10101010);
    }
    digitalWrite(row + 3, HIGH);
    delayMicroseconds(150);
    digitalWrite(row + 3, LOW);
    isEvenRow = !isEvenRow;
  }
}

Well, neat! It worked? Playing with the SPI_CLOCK_DIv variable made the loop even quicker, down to the point where I had to up the delay at the end so that the LEDs would illuminate for a longer timeframe. SPI_CLOCK_DIV determines the speed at which the data is sent out, dividing the main 16mhz clock on the Arduino. DIV2 halves it (8mhz), DIV4 = 1/4 = 4mhz, DIV8 = 1/8 = 2mhz, DIV16 = 1/16 = 1mhz, etc...

Anyway, the test pattern was great... but I needed to output text, in rows of bytes!?

What About Text?

So this is where it gets interesting. My font array has 7 rows of 5 pixels per font. I then have up to 6 characters per LED display unit, equalling a total of 30 bits each. Of course, there's 8 bits to a byte, so the neatly rounded-up value would be 32 bits. I could therefore send out 4 bytes to get each row transmitted over the wire. But do be warned, if you start the data on the first bit out, it'll 'run off the edge' of the screen as the screen is only 30 bits wide...

As you can see, the text to be displayed is "TEST!" in the cute Arduino 5x7 font. Our display is 30 bits wide, so there's two columns of blank to the left. After the 2 pixel offset, we start throwing out each row of data. This won't be overly-easy either, as we need to get the rows of bits per character per font.

So, based on the current pixel to be pushed out, firstly get the character from the string array. We then need to look up the font pixel array for this character, only getting the relevant row that we're trying to display. Once we have that, we can then just send the bits out to a temporary array of bytes. As that we'll only have 5 bits per font line row, we'll actually need to merge multiple characters into each byte of the array. If we then want to scroll text, we'll have to put a little more effort in. This isn't overly different from when we were sending the data out bit-by-bit; we're just stuffing it into bytes now, making sure we get everything into the correct byte in the output array.

I jammed this calculation into one row below and the output was actually quite surprising! Mainly in the fact that it actually worked!

#include <SPI.h>
#include "myfont.h"

String serialString = "TEST! ";
const int BUFFER_LENGTH = 4;
const int DISPLAY_LENGTH = 30;
byte* datarow = new byte[BUFFER_LENGTH];

void setup() {
  for (int p = 0; p <= 12; p++) {
    pinMode(p, OUTPUT);
    digitalWrite(p, LOW);
  }
  SPI.begin();
  SPI.setDataMode(SPI_MODE2);
  SPI.setBitOrder(LSBFIRST);
  SPI.setClockDivider(SPI_CLOCK_DIV128);
}

void loop() {
  for (int row = 0; row < 7; row++) 
  {
    for (int col = 0; col < DISPLAY_LENGTH; col++) {
      datarow[((col + ((BUFFER_LENGTH * 8) - DISPLAY_LENGTH)) / 8)] |= 
        ((font[serialString[col / 5]][col % 5] & (1 << (6 - row))) 
          ? 1 : 0) << ((col) % 8);
    }
    for (int col = 0; col < BUFFER_LENGTH; col++) SPI.transfer(datarow[col]);
    digitalWrite(row + 3, HIGH);
    delayMicroseconds(500);
    digitalWrite(row + 3, LOW);
  }
}

Everything was actually working nicely... but then I tried to extend it out to 4 displays. 120 pixels out, even at the fastest SPI, caused severe flickering. It dawned on me that I was doing some pretty hefty lifting in my display loop, presumably slowing down the drawing. It's actually a critical loop and it seems that any processing will slow down the line drawing, causing the flickering I was seeing.

With this problem in mind, it became apparent that I'd need to move the 'buffer' calculation out of the main loop. I quickly tested this theory by moving it to a once-off in the setup procedure.

#include <SPI.h>
#include "myfont.h"

String serialString = "TEST! ";
const int BUFFER_LENGTH = 4;
const int DISPLAY_LENGTH = 30;
const int DISPLAY_HEIGHT = 7;
byte* datarow = new byte[DISPLAY_HEIGHT][BUFFER_LENGTH];
void setup() {
  for (int p = 0; p <= 12; p++) {
    pinMode(p, OUTPUT);
    digitalWrite(p, LOW);
  }
  SPI.begin();
  SPI.setDataMode(SPI_MODE2);
  SPI.setBitOrder(LSBFIRST);
  SPI.setClockDivider(SPI_CLOCK_DIV128);

  for (int row = 0; row < DISPLAY_HEIGHT; row++) 
  {
    for (int col = 0; col < DISPLAY_LENGTH; col++) {
      datarow[row][((col + ((BUFFER_LENGTH * 8) - DISPLAY_LENGTH)) / 8)] |= 
        ((font[serialString[col / 5]][col % 5] & (1 << ((DISPLAY_HEIGHT - 1) - row))) 
          ? 1 : 0) << ((col) % 8);
    }
  }
}

void loop() {
  for (int row = 0; row < DISPLAY_HEIGHT; row++) 
  {
    for (int col = 0; col < BUFFER_LENGTH; col++) SPI.transfer(datarow[row][col]);
    digitalWrite(row + 3, HIGH);
    delayMicroseconds(500);
    digitalWrite(row + 3, LOW);
  }
}

My text was static, so I now only built the buffer once. Would you believe it? The rendering was beautiful! I couldn't even notice a flicker. This was then short-lived when I realised I needed to scroll text. Again, there was no need to rebuild the buffer on each loop, so I wrote a procedure and only called it when there was text to move; defined by a delayed scroll rate.

This worked nicely, but there was an obvious jarring delay as the text shifted along. You could feel it re-processing in-between the character movements. I was still happy with it! I went ahead and reconfigured my initial display setup to use two Arduinos: one for the remocon+wifi and the other to receive serial data and light this dispalay.

But Wait, Why Don't I Build The Entire Text Buffer?

Whilst writing this up, it occurred to me that I was only rebuilding the buffer for the text that was visible on the display, not the entire string. If I had the full text buffer already created somewhere in memory, I then would only need to get SPI to send out the data from a specific bit offset. But, how do you start sending data via SPI from half-way through a byte? My fonts/characters were 5 bits wide and each byte had 1.5ish characters in them! I considered just shifting the pointer along the bytes, but the scrolling would be horrible.

So, I want to send out data from half way through a byte? Let's just borrow that table from above again:

Officially, on the 3rd loop of scrolling, I would want to send out the green bits. I'd also then want to send out five zeroes at the other end so that the scrolling text trails off and doesn't wrap around. I can't tell SPI to start at the 4th bit of a byte? So I'll need to create a new buffer and populate this as quickly as possible, line by line, just as we're about to render it. Thanks to the glory of the internet, there's always someone else who wants to do something similar. Effectively we have an array of bytes (unsigned chars, meaning we get all 8 bits) and we'll need to shift the first one and append the next one. It'll be very much like a catepillar crawling along.

So, if we build the full text buffer array when the string comes in, we can then push the relevant chunk into the display buffer. There'll be no font calculations, just bit for bit copying, and this will hopefully still be fast enough. We can even make sure we only do it line by line, and only as required... if we're on a byte boundary, then we don't need to shift anything!

I mentioned shifting above, and that's what we'll need to do to get the chunks of bytes into the final display buffer. If you look at the table above, you'll notice I coloured the cells in light green and dark green. The first 3 bits are light green and come from the 'end' of the first byte of the text buffer. The next 5 bits are from the 'start' of the second byte in the text buffer. This chain then continues on as we shift the bits. The >> and << C operators will assist us with this task as that's exactly what they're designed to do: shift bits in a variable in a certain direction by a certain amount. Let's try some pseudo-code first....

define a string buffer that has the string to display: "TEST!";
build a buffer for this string: stringBitBuffer[7][4];
define a display buffer that is enough for 1 panel: displayBuffer = byte[7][4];
if our scroll offset is a multiple of 8
  foreach stringBitBuffer[scrolloffset /8] SPI.writeRow();
  write extra zeroes at the end.
else if our scroll offset is NOT a multiple of 8
  foreach byte in stringBitBuffer starting from (scrollOffset/8)
    shift the first block of the text buffer right scrollOffset times
    make anything to the left of the text in that byte zero.
    move the resulting value into the current displayBuffer byte.
    get the next text buffer byte. 
    take the first (8 - scrollOffset) bits and add them to the current displayBuffer byte
  next
end if

Here's an animation showing what bits we want where...

The table above shows the 8 frames that need to be copied over to the framebuffer between bytes. Once the data has shifted a whole byte to the left, then the drawing routine just needs to start from the next byte, there's no need to send out the initial bytes if they're offscreen. We need to copy the bits in the main area of the table into the buffer bytes listed on top. I've added a new row at the bottom which indicates the first bit/byte of the text buffer (TB). With the duplicated Byte row underneath, you can see how we'll need a part of one text buffer and the rest of the next to fill the gaps.

Back to that forum post I mentioned above, and the description I gave about bit shifting, we can build up our buffer. We need to, based on the scroll offset, take the first (8 - offset) bits from every byte and then the 'offset' bits from the following byte. i.e. if the offset was 3, we want the last 5 bits of byte 1 and the first three of byte 2. If we're on the last byte, then we just fill the end with zeroes. The AND masks for row 2 are shown below. Row 1 is nearly all bits set... so wouldn't be a good test case.

And then we just need to shift the first 5 bits to the left... as we're stored in MSB.

#include <SPI.h>
#include "myfont.h"

int ccol = 0;
byte datarow[7][60];
int displayLength = 120;
int bytesRequired = (displayLength + (displayLength % 8) + 8) / 8;
int totalPixels = bytesRequired * 8;
byte frameBuffer[7][16];

unsigned long last_time = millis();
unsigned long scroll_time = millis();
unsigned long scroll_pause = millis();
unsigned long next_text_time = millis();
int charOffset = 0, loop_count = 0, charcol = 0;
String displayString;

void buildTextBuffer() {
  bytesRequired = ((displayString.length() * 5) / 8) + ((displayString.length() * 5) % 8 == 0 ? 0 : 1);
  totalPixels = bytesRequired  * 8;
  for (int row = 0; row < 7; row++) 
  {
    for (int b = 0; b < bytesRequired; b++) datarow[row][b] = 0;
    for (int col = 0; col < totalPixels; col++) {
      ccol = col + 8;
      datarow[row][ccol / 8] |= 
        ((font[displayString[col / 5]][col % 5] & (1 << (6 - row))) 
          ? 1 : 0) << (ccol % 8);
    }
  }
}

void buildFrameBuffer(int offset = 0) {
  int tOffset = offset / 8;
  int cOffset = offset % 8;
  int bb = 0;
  for (int row = 0; row < 7; row++) 
  {
    for (int b = 0; b < 16; b++) {
      bb = b+tOffset;
      frameBuffer[row][b] = (datarow[row][bb] >> cOffset);
      if (cOffset > 0) {
        if (bb <= bytesRequired) {
          frameBuffer[row][b] |= (datarow[row][bb + 1] << (8 - cOffset));
        }
      }
    }
  }
}

void setup() {
  Serial.begin(9600);
        
  for (int p = 0; p <= 12; p++) {
    pinMode(p, OUTPUT);
    digitalWrite(p, LOW);
  }
  
  SPI.begin();
  SPI.setDataMode(SPI_MODE3);
  SPI.setBitOrder(LSBFIRST);
  SPI.setClockDivider(SPI_CLOCK_DIV64);

  displayString = "Warming up ... ";
  buildTextBuffer();
  buildFrameBuffer(0);
}

String newString = "";
void getNextIndex() {
  for (int i = 0; i < 7; i++)
    for (int x = 0; x < 60; x++)
      datarow[i][x] = 0;
      
  if (Serial.available()) {
    newString = Serial.readStringUntil('\n');
    if (!newString.equals(displayString)) {
      displayString = newString;
      next_text_time = millis();
    }
  } else {
    displayString = "";
  }
  if (displayString.length() > 0) buildTextBuffer();
  buildFrameBuffer(0);
}

void loop() {  
  for (int row = 0; row < 7; row++) 
  {
    for (int col = 0; col < 16; col++) SPI.transfer(frameBuffer[row][col]);
    digitalWrite(row + 3, HIGH);
    delayMicroseconds(150);
    digitalWrite(row + 3, LOW);
  }

  if ((millis() - scroll_pause > 2500)) {
    if ((displayString.length() > (displayLength / 5))) {
      if (millis() - scroll_time > 30) {
        charOffset++;
        if ((charOffset / 5) > displayString.length()) {
          charOffset = 0;
          loop_count++;
          scroll_pause = millis();
          if (millis() - next_text_time > 5000) getNextIndex();
        }
        scroll_time = millis();
        buildFrameBuffer(charOffset);
      }
    } else {
      loop_count++;
      if (millis() - next_text_time > 5000) {
        getNextIndex();
        loop_count = 0;
      }
      charOffset = 0;
    }
  }
}

And it all actually worked... beautiful scrolling.

Further Possible Optimisations

We're sending the text into this Arduino from another Arduino over SoftSerial. Why send text when I could send the framebuffer bits instead? I wonder if sending (string-length * 5 * 7 * 8) bits would be quicker than the initial string and calculations on the secondary Arduino's side? Should I test it? Maybe later...

Arduino Uno WiFi + TV Remote + LED Sign

Over the weekend, whilst watching TV, I realised that I needed some kind of sign to work out when trains were passing my apartment. There's a railway line right next to me and around four-or-more services pass daily. I'd prefer to be notified in advance and, since I have data sources to tell me when trains are about to come by, I want to make the most of it. Currently that's done by RSS both on my phone and PC, but the whole point is that I'm watching TV and don't want to have to be checking my phone every 2 seconds.

After getting the red LED marquee to work on the old ISA I/O card, I decided it was time to get it plugged into the Arduino. I actually already have an Arduino Uno Wifi next to my TV, which I've recently been using to translate my TV remote control codes (similar to what I did with the old B&O TV) into Yamaha remote control codes for control of volume with one remote. Adding the LED marquee would let me know when a button was successfully pressed ... and it would also tell me when trains were coming past!

I hadn't actually worked with the WiFi on the Arduino yet, so that was a bit of an experience. The RF side of things was working fine and I had already worked out the wiring requirements for the LED display thanks to the old I/O card.

Getting the ESP8266 Online

The Arduino Uno Wifi is an Arduino Uno board with an embedded ESP8299 microcontroller. The basic idea is that the two units talk serial to each other, when configured to do so. If you want to program one, or the other, you have to adjust the serial communications path on the board via a tiny row of switches. Jaycar has a minimal instruction sheet here, but it's enough to get you started. From the information, please set your Arduino Uno Wifi's tiny switch row to: 1:OFF,2:OFF,3:OFF,4:OFF,5:ON,6:ON,7:ON,8:OFF.

I would want this unit to act by itself as both a web server and web requestor, so I needed to find code for both. Before this we need to set up the Arduino IDE for use with the ESP8266. I found a great tutorial here, but otherwise, the steps are pretty simple. Open the Arduino IDE, go to Preferences and then paste this URL into to the additional boards list: http://arduino.esp8266.com/stable/package_esp8266com_index.json.

Once done, go to the board manager and search for ESP8266...

Install the relevant row and then restart the Arduino IDE. From here, you should have a new selection list in the boards drop-down...

...and then, once selected, a hideous amount of new options to adjust the programming!

The main advice is to not touch any of them, apart from the serial port. You'll need to work out which one your Arduino is connected to, but depending on if you even had serial ports in the first place, it should be pretty obvious. I'm working on a Mac Mini and my Arduino was connected via cu.usbserial-1460. From here, open the Serial Monitor and hit reset on the Arduino.

What garbage is that? I then started from the slowest and tested each baud... turns out if you set to 74880 then you'll get the following...

What a weird port number? It seems that just the initial header information is at that BAUD rate. Afterwards, it'll be at whatever you've set in the code you upload.

Running A Web Server

So, where were we? We have a connected device, so let's grab the first example from here and compile it. After installing the ESP8266 libraries, this code should compile with zero issues. If you happen to have any, then write a comment below, providing as much detail as possible. If it's worked, then check the code and adjust the Wifi SSID and password to match your local AP. Hit upload and cross your fingers and toes... hopefully you'll get the following output:

Yes! Now, you need to switch OFF switch 7 on the little switch header on the Arduino. This pin enables the programming mode and we now want it in the run mode. Note also that we've now set the baud to 115200, so change your serial monitor to that too... Do you get the following after hitting the reset key on the Arduino?

Associated! And with a nice IP! So what does the web browser say?

WIN!

HTTPS Web Requests with the ESP8266

The basic ESP8266 examples use WiFiClient, but this only works with HTTP requests. My server runs over HTTPS, so we need to use WifiSecureClient for secure web requests. To call an HTTPS service, you'll need to get the fingerprint from the SSL certificate. Browse to the site in a regular web browser and check out the properties of the lock next to the URL in the address bar. Somewhere in there you'll find the required data.

And from there, edit the WifiClient Example to carry out an SSL request using these instructions:

#include <esp8266wifi .h>

const char* ssid = "********";
const char* password = "********";
const char* host = "api.github.com";
const char* fingerprint = "CF 05 98 89 CA FF 8E D8 5E 5C E0 C2 E4 F7 E6 C3 C7 50 DD 5C";

void setup()
{
  Serial.begin(115200);
  Serial.println();

  Serial.printf("Connecting to %s ", ssid);
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED)
  {
    delay(500);
    Serial.print(".");
  }
  Serial.println(" connected");
}


void loop()
{
  WiFiClientSecure client;
  Serial.print("[connecting to ");
  Serial.println(host);
  if (client.connect(host, 443)) {
    Serial.println("connected]");
    if (client.verify(fingerprint, host)) {
      Serial.println("[certificate matches]");
    } else {
      Serial.println("[certificate doesn't match]");
    }
    Serial.println("[Sending a request]");
    client.print(String("GET /") + " HTTP/1.1\r\n" +
                 "Host: " + host + "\r\n" +
                 "Connection: close\r\n" +
                 "\r\n");
    Serial.println("[Response:]");
    while (client.connected() || client.available())
    {
      if (client.available())
      {
        String line = client.readStringUntil('\n');
        Serial.println(line);
      }
    }
    client.stop();
    Serial.println("\n[Disconnected]");
  }
  else
  {
    Serial.println("connection failed!]");
    client.stop();
  }
  delay(5000);
}

And hopefully, you'll get something similar to the following in your serial terminal.

{ll⸮⸮|⸮d⸮|⸮d⸮b|⸮⸮⸮⸮s⸮b⸮c⸮⸮'o⸮dg'⸮⸮⸮cp⸮⸮dsdrlx⸮o⸮⸮d⸮⸮co⸮|l⸮⸮c⸮⸮go⸮d⸮⸮$`⸮'o$`gs⸮⸮⸮ob⸮$;⸮⸮gc⸮l⸮⸮dl⸮⸮d`⸮⸮'⸮

Connecting ...
scandone
........scandone
.scandone
state: 0 -> 2 (b0)
state: 2 -> 3 (0)
state: 3 -> 5 (10)
add 0
aid 1
cnt 

connected with COMEGETSOME, channel 6
dhcp client start...
ip:192.168.1.121,mask:255.255.255.0,gw:192.168.1.1

Connected to COMEGETSOME
IP address:	192.168.1.121
mDNS responder started

connecting to api.github.com
Using fingerprint '59 74 61 88 13 CA 12 34 15 4D 11 0A C1 7F E6 67 07 69 42 F5'
requesting URL: /repos/esp8266/Arduino/commits/master/status
request sent
headers received
esp8266/Arduino CI has failed
reply was:
==========
{"state":"pending","statuses":[],"sha":"1e17dddd895883806c9bfd3dd7b7042aa813d94f","total_count":0,"repository":{"id":32969220,"node_id":"MDEwOlJlcG9zaXRvcnkzMjk2OTIyMA==","name":"Arduino..."}
==========
closing connection

Nice, we can successfully pull data from a HTTPS source. Let's rig up a polling system to get the data each minute.

Polling for Data + Extra Buttons

So, we're able to pull data from the internet, but I also want this unit to be a mini webserver. If I used Delay() to spread out the web-polling calls, then I'd halt the entire unit. We can't have it wait when it needs to be handling button presses from the web interface. Instead, we'll take a snapshot of the current internal millisecond counter using millis() and then check how many milliseconds have passed to work out if we should try and grab new data.

#include <ESP8266WiFi.h>
#include <WiFiClient.h>
#include <ESP8266WiFiMulti.h> 
#include <ESP8266mDNS.h>
#include <ESP8266WebServer.h>   // Include the WebServer library

ESP8266WiFiMulti wifiMulti;     // Create an instance of the ESP8266WiFiMulti class, called 'wifiMulti'
ESP8266WebServer server(80);    // Create a webserver object that listens for HTTP request on port 80

void handleRoot();              // function prototypes for HTTP handlers
void handleNotFound();
void handleRemote();
unsigned long lastMillis;
const char* host = "trainfinder.otenko.com";
const int httpsPort = 443;
const char fingerprint[] PROGMEM = "BC 9C 56 D5 8E 7A FE CC 0C F7 A2 21 1F C5 7B 9A C0 FA 15 41";
String last_line = "";

void setup(void) {
  Serial.begin(9600);
  delay(100);
  Serial.println("\n!START");
  wifiMulti.addAP("COMEGETSOME", "blahblahblah");
  while (wifiMulti.run() != WL_CONNECTED) { delay(250); }
  Serial.println(WiFi.SSID());
  Serial.println(WiFi.localIP());

  server.on("/", HTTP_GET, handleRoot);
  server.on("/buttons", HTTP_POST, handleRemote);
  server.onNotFound(handleNotFound);
  server.begin();
  
  lastMillis = millis();
}

void loop(void){
  server.handleClient();
  if ((millis() - lastMillis) > (60 * 1000)) {
    WiFiClientSecure client;
    client.setFingerprint(fingerprint);
  
    if (!client.connect(host, httpsPort)) {
      Serial.println("!FAILED");
    } else {
      String url = "/some/cool/url";    
      client.print(String("GET ") + url + " HTTP/1.1\r\nHost: " + host + "\r\nConnection: close\r\n\r\n");
      while (client.connected()) {
        String line = client.readStringUntil('\n');
        if (line == "\r") break;
      }
      bool foundLastLine = false;
      String line = "";
      while(client.available()) {   
        line = client.readStringUntil('\n');
        if (last_line.equals("") || foundLastLine) {
          Serial.println(line);
          last_line = String(line);
          delay(6500); //space out the data to the Arduino
        }
        if (last_line.equals(line)) {
          foundLastLine = true;
        }
      }
      if (!foundLastLine) last_line = line;
    }
    lastMillis = millis();
  }
}

void handleRoot() {
  server.send(200, "text/html", "<form action='/buttons' method='POST'>" \
      "<input type='submit' name='pwr_plz' value='PWR!'/>" \
      "<input type='submit' name='vol_dn' value='VOL-'/>" \
      "<input type='submit' name='vol_up' value='VOL+'/>" \ 
    "</form><br/><br/>Last Line: <pre>" + last_line + "</pre>");   
}

void handleRemote() {
  if (server.hasArg("vol_up")) {
    Serial.println("!VOLUP");
  } else if (server.hasArg("vol_dn")) {
    Serial.println("!VOLDN");
  } else if (server.hasArg("pwr_plz")) {
    Serial.println("!POWER");
  }
  server.sendHeader("Location","/");
  server.send(303);
}

void handleNotFound(){
  server.send(404, "text/plain", "404: Not found");
}

Just for fun, here's the first shot of me writing the code. Instead of 'text/html', I was still outputting 'text/plain' and, well, as expected, got the following...

After fixing that, the screenshot below was taken. Of course, it doesn't match the code above, but you get the idea.

The basic idea is to print out the raw html to the browser. From here there's a form that has the buttons which post back to the ESP8266. The controller then sees the post, sends the command via Serial to the Arduino and then redirects the user back to the main form.

Now to get the Arduino to consume it.

ESP8266 to Arduino

We've currently had the ESP8266 piped directly out the USB Serial port. Now we want it to talk to the Arduino internally. To do this, the tiny onboard switch needs to have all switches off except for 1 and 2. This then routes the RX/TX directly from the ESP8266 to the Arduino. From here though, the unit becomes a black-box. We have no idea what the ESP8266 is saying to the Arduino, and vice-versa, as the connection via the USB serial port has been severed. We can only hope that the data we're sending through is what we want!

If you check the shots above of the data the ESP8266 sends, you can see that we want to skip a bit of the earlier stuff. To do this, I defined a "!START" command which the Arduino should look for before bothering to deal with any of the data from the serial channel. From there, anything starting with a "!" is a command, whereas anything else is to be printed to the LED Display.

Oh yeah, that TV Remote bit

I previously had an amplifier that ran over ARC over HDMI and also received controls... but half the time it'd switch on to Optical and just would not switch back to ARC without a lot of screwing around. No amount of power cycling, cable switching or button pressing would get it back. So it went to the farm. Instead, I purchased a beautiful old Yamaha amp for AUD$5 and then realised I had to get up and control it by hand. Not optimal, so I search google for the remote codes.... they didn't exist, so I search eBay for a spare remote... they didn't exist... but one guy in Spain had a company that programmed third-party remotes and had one for this amp!

It arrived, and I plugged it through the same code I used for the B&O TV. Over the serial, the codes and bits were reported, showing me that it used the "NEC" protocol. I recorded the codes for the buttons I wanted. I then did the same with my Sony TV Remote for buttons that I wanted to re-purpose. The goal was to have the TV Volume buttons also trigger the amp volume. This worked nicely, via an external IR LED that sits in front of the amp.

Of course, the TV volume still shifts, so I made the amp volume shift 4 times per button press... meaning the surround is always louder than the TV. Doesn't hurt to have a little bit of tinny sound from the TV also... a second center channel?

To make everything compile, install Ken Shirrif's IRremote library, available via the Arduino GUI...

The code is in the next segment.

Hooking up the LED Sign

I've used a font found online, but it seems that Arduino has one built-in? 5x7 font. Either way, the 3 data lines and 7 row-enable pins need to find homes in the Arduino digital IO pins. Make sure you don't use digital pin 0 or 1 as that's the serial channel to the ESP8266. I found this out later below...

Once hooked up, just use a similar loop process as per any LED display: Disable all rows, send out the columns to be lit for the first row, enable that row and quickly disable it. Then do the same for the next 6 rows. As quickly as possible! More information on lighting this sign is over here.

#include <IRremote.h>
#include "myfont.h"
#include "digitalWriteFast.h"

int RECV_PIN = 13;
IRrecv irrecv(RECV_PIN);
decode_results results;
IRsend irsend;

void setup()
{
  Serial.begin(9600);
  irrecv.enableIRIn(); // Start the receiver

  pinModeFast(0, OUTPUT);
  digitalWriteFast(0, LOW);
  pinModeFast(2, OUTPUT);
  digitalWriteFast(2, LOW);
  pinModeFast(12, OUTPUT);
  digitalWriteFast(12, LOW);

//matrix led row drivers
  for (int p = 0; p <= 11; p++) {
    pinModeFast(p, OUTPUT);
    digitalWriteFast(p, LOW);
  }
}

unsigned long last_time = millis();
unsigned long scroll_time = millis();
unsigned long scroll_pause = millis();
bool is_off = true;
int loop_count = 0;
String serialString = " Warming up..";
bool found_start = false;
int charOffset = 0;
int ccol;

void loop() {
  if (irrecv.decode(&results)) {    
    loop_count = 0;
    switch (results.value) {
      case 4841:
      case 0xA90:
        irsend.sendNEC(0xDE21807F, 32);
        if (is_off) {
          serialString = " Power ON!";
          is_off = false;
        } else {
          serialString = " GOOD BYE!";
          is_off = true;
        }
        break;
      case 0x490:
        irsend.sendNEC(0xDE21B04F, 32);
        delay(50);
        irsend.sendNEC(0xDE21B04F, 32);
        delay(50);
        irsend.sendNEC(0xDE21B04F, 32);
        serialString = " VOLUME UP!";
        break;
      case 0xC90:
        irsend.sendNEC(0xDE21708F, 32);
        delay(50);
        irsend.sendNEC(0xDE21708F, 32);
        delay(50);
        irsend.sendNEC(0xDE21708F, 32);
        serialString = " VOLUME DOWN!";
        break;
      case 0xFFFFFFFF:
        irsend.sendNEC(0xFFFFFFFF, 32);
        break;
        //sony teletext red
      case 21225:
        irsend.sendNEC(3726721215, 32);
        serialString = " INPUT!";
        //sony teletext green
      case 13033:
        irsend.sendNEC(3726717135, 32);
        serialString = " MODE UP!";
        break;
        //sony teletext yellow
      case 29417:
        irsend.sendNEC(3726735495, 32);
        serialString = " MODE DOWN!";
        break;
      default:
        Serial.println(results.value);
        Serial.println(results.bits);
        break;
    }
    delay(250);
    irrecv.enableIRIn();
    irrecv.resume(); // Receive the next value
  }

  if (charOffset == 0 && Serial.available() && ((millis() - last_time) > 4000)) {
    last_time = millis();
    serialString = Serial.readStringUntil('\n') + '\0';
    loop_count = 0;
    scroll_pause = millis();
  }
  
  //draw LEDs
  for (int row = 0; row < 7; row++) 
  {
    for (int col = 0; col < 90; col++) {
      if (((col / 5) + charOffset) < 60) {
        ccol = (charOffset * 5) + col;
        digitalWriteFast(2, ((font[serialString[ccol / 5]][ccol % 5] & (1 << (6 - row))) > 0) ? HIGH : LOW);
      } else {
        digitalWriteFast(2, LOW);
      }
      digitalWriteFast(12, HIGH);
      delayMicroseconds(5);
      digitalWriteFast(12, LOW);
    }
    digitalWriteFast(row + 5, HIGH);
    delayMicroseconds(75);
    digitalWriteFast(row + 5, LOW);
  }
  
  if ((millis() - scroll_pause > 2500)) {
    if ((serialString.length() > 18)) {
      if (millis() - scroll_time > 180) {
        charOffset++;
        if (charOffset > serialString.length()) {
          charOffset = 0;
          loop_count++;
          scroll_pause = millis();
          if (loop_count > 10) serialString = "";
        }
        scroll_time = millis();
      }
    } else {
      loop_count++;
      if (loop_count > 100) serialString = "";
      charOffset = 0;
    }
  }
}

Overloading Arduino Pins

Anyone watching above, or in the previous post where I hooked the display up to the old PC, was probably screaming in their seats. It's well known that you should NOT draw too much current through the pins on any microprocessor. As soon as I started hooking up more modules onto this unit, things started going haywire and it quickly dawned on me. My seven line-enable pins were being brought LOW to enable the rows... but these were actually working as the single GND conduit for the entire bloody LED display!

I quickly referred to the previous module that ran this sign and saw there was a 74LS145 binary counter in between the Atmel Microcontroller and the 7 enable rows. A quick review of the datasheet shows that this chip can 'sink' up to 15v and is 'great for driving LEDs'. Right... they were properly routing the GND wires away from the microcontroller and using the 145 as a set of transistors. They even saved 3 data lines in the process! I only had a ULN2003 (which would still use 7 lines) and there were no 74LS145 to be found at Jaycar!

This now directed the current away from the microcontroller... but didn't enhance the brightness.

Speeding up the Refresh Rate

It occurred to me that my refresh rates weren't anywhere near as good as the original controller that the sign contained. My loop is busy checking clocks, serial ports and mucking around, so it was never going to actually go fast enough. The code above could run a single segment OK, but started to scale worse as the bit shifting got longer. One trick, which is actually already implemented in the source above is to use an external library to make the digital pin manipulation faster.

With this implemented, there was less flickering and I found that three segments (90 LEDs per row) looked good. One final note is that you shouldn't really use pins 0 or 1 if you've got the serial enabled. They actually present the TX/RX lines and the serial data and the serial data to the Arduino is visible on them. In this case, it was the serial data to/from the ESP8266. In the case below, I had the three data lines required to drive the display on digital pins 0,1,2...

Shifting those to spare pins higher...

Yessssss! It works really nicely.