Subscribe via RSS
5Dec/190

More (Bit)Bang For Your Buck

Arduinos are a great hobbyist platform for projects, but you'll often encounter speed and memory issues when you try to squeeze in too many features. Fortunately, being an open source system, one can easily bypass the standard libraries and push these little units as hard as possible. Of course, once you're past the warranties and disclaimers, you're out on your own.

I've previously had to jump out of the IDE and use 'faster' code whilst using a Dreamcast Controller for my Model Railway, but that was a little more advanced than what we're about to do here. Today's goal is to bitbang the data stream to the Red LED Sign as quickly as possible. If you check that post, you'll see I'm already using digitalWriteFast, but it seems that we can go even quicker with the SPI library.

What's SPI, you ask? To steal from Arduino: Serial Peripheral Interface (SPI) is a synchronous serial data protocol used by microcontrollers for communicating with one or more peripheral devices quickly over short distances. As it turns out, that's exactly what we want to do! We want the display to be a slave and, well, we're just going to hack the right RX/TX lines together to get the data to flow. A fellow Arduino-er has done this here whilst interfacing with a 74HC595 and provides a few hints as to problems we might run in to.

Actually, now that I think of it, the data is to be sent out in bytes, but my screen is no where near the order of a clean multiplication of bytes... will have to work on that. Also... the data I read is the top 'line' of each letter, it's not a clean char in any memory array. Hmmm...

Test Patterns

Without pulling apart my currently-hacked-together-and-functional-display, I used another Uno I had lying around and hooked it up to the two spare LED panels. The whole display came with 5 panels, but I wasn't able to smoothly run them all at once, so these two were not in use.

The hook-up is pretty straight forward. The Wikipedia article on SPI gives a little more information on the bus and, more specifically, dictates which wires we need to use. We'll need the data output wire and the clock, known as MOSI, or Master Out-Slave In, and SCLK, the serial clock. Note that some devices read the data pin when the clock transitions from LOW to HIGH, and some vice-versa. Fortunately, this can all be configured via the SPI library.

With the following code, I managed to get a test pattern on the display...

#include <SPI.h>
void setup() {
  for (int p = 0; p <= 12; p++) {
    pinMode(p, OUTPUT);
    digitalWrite(p, LOW);
  }
  SPI.begin();
  SPI.setDataMode(SPI_MODE2);
  SPI.setBitOrder(LSBFIRST);
  SPI.setClockDivider(SPI_CLOCK_DIV128);
}

bool isEvenRow = false;
void loop() {
  isEvenRow = false;
  for (int row = 0; row < 7; row++) 
  {
    if (isEvenRow) {
      SPI.transfer(B01010101);
      SPI.transfer(B01010101);
      SPI.transfer(B01010101);
      SPI.transfer(B01010101);
    } else {
      SPI.transfer(B10101010);
      SPI.transfer(B10101010);
      SPI.transfer(B10101010);
      SPI.transfer(B10101010);
    }
    digitalWrite(row + 3, HIGH);
    delayMicroseconds(150);
    digitalWrite(row + 3, LOW);
    isEvenRow = !isEvenRow;
  }
}

Well, neat! It worked? Playing with the SPI_CLOCK_DIv variable made the loop even quicker, down to the point where I had to up the delay at the end so that the LEDs would illuminate for a longer timeframe. SPI_CLOCK_DIV determines the speed at which the data is sent out, dividing the main 16mhz clock on the Arduino. DIV2 halves it (8mhz), DIV4 = 1/4 = 4mhz, DIV8 = 1/8 = 2mhz, DIV16 = 1/16 = 1mhz, etc...

Anyway, the test pattern was great... but I needed to output text, in rows of bytes!?

What About Text?

So this is where it gets interesting. My font array has 7 rows of 5 pixels per font. I then have up to 6 characters per LED display unit, equalling a total of 30 bits each. Of course, there's 8 bits to a byte, so the neatly rounded-up value would be 32 bits. I could therefore send out 4 bytes to get each row transmitted over the wire. But do be warned, if you start the data on the first bit out, it'll 'run off the edge' of the screen as the screen is only 30 bits wide...

Bytes 00 01 02 03
Pixels -- -- 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
Row 0 00 00 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 00 00 01 00 00 00 00 00 00 00
Row 1 00 00 00 00 01 00 00 01 00 00 00 00 01 00 00 00 00 00 00 01 00 00 00 00 01 00 00 00 00 00 00 00
Row 2 00 00 00 00 01 00 00 01 00 00 00 00 01 00 00 00 00 00 00 01 00 00 00 00 01 00 00 00 00 00 00 00
Row 3 00 00 00 00 01 00 00 01 01 01 01 01 01 01 01 01 01 00 00 01 00 00 00 00 01 00 00 00 00 00 00 00
Row 4 00 00 00 00 01 00 00 01 00 00 00 00 00 00 00 00 01 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00
Row 5 00 00 00 00 01 00 00 01 00 00 00 00 00 00 00 00 01 00 00 01 00 00 00 00 01 00 00 00 00 00 00 00
Row 6 00 00 00 00 01 00 00 01 01 01 01 01 01 01 01 01 01 00 00 01 00 00 00 00 01 00 00 00 00 00 00 00

As you can see, the text to be displayed is "TEST!" in the cute Arduino 5x7 font. Our display is 30 bits wide, so there's two columns of blank to the left. After the 2 pixel offset, we start throwing out each row of data. This won't be overly-easy either, as we need to get the rows of bits per character per font.

So, based on the current pixel to be pushed out, firstly get the character from the string array. We then need to look up the font pixel array for this character, only getting the relevant row that we're trying to display. Once we have that, we can then just send the bits out to a temporary array of bytes. As that we'll only have 5 bits per font line row, we'll actually need to merge multiple characters into each byte of the array. If we then want to scroll text, we'll have to put a little more effort in. This isn't overly different from when we were sending the data out bit-by-bit; we're just stuffing it into bytes now, making sure we get everything into the correct byte in the output array.

I jammed this calculation into one row below and the output was actually quite surprising! Mainly in the fact that it actually worked!

#include <SPI.h>
#include "myfont.h"

String serialString = "TEST! ";
const int BUFFER_LENGTH = 4;
const int DISPLAY_LENGTH = 30;
byte* datarow = new byte[BUFFER_LENGTH];

void setup() {
  for (int p = 0; p <= 12; p++) {
    pinMode(p, OUTPUT);
    digitalWrite(p, LOW);
  }
  SPI.begin();
  SPI.setDataMode(SPI_MODE2);
  SPI.setBitOrder(LSBFIRST);
  SPI.setClockDivider(SPI_CLOCK_DIV128);
}

void loop() {
  for (int row = 0; row < 7; row++) 
  {
    for (int col = 0; col < DISPLAY_LENGTH; col++) {
      datarow[((col + ((BUFFER_LENGTH * 8) - DISPLAY_LENGTH)) / 8)] |= 
        ((font[serialString[col / 5]][col % 5] & (1 << (6 - row))) 
          ? 1 : 0) << ((col) % 8);
    }
    for (int col = 0; col < BUFFER_LENGTH; col++) SPI.transfer(datarow[col]);
    digitalWrite(row + 3, HIGH);
    delayMicroseconds(500);
    digitalWrite(row + 3, LOW);
  }
}

Everything was actually working nicely... but then I tried to extend it out to 4 displays. 120 pixels out, even at the fastest SPI, caused severe flickering. It dawned on me that I was doing some pretty hefty lifting in my display loop, presumably slowing down the drawing. It's actually a critical loop and it seems that any processing will slow down the line drawing, causing the flickering I was seeing.

With this problem in mind, it became apparent that I'd need to move the 'buffer' calculation out of the main loop. I quickly tested this theory by moving it to a once-off in the setup procedure.

#include <SPI.h>
#include "myfont.h"

String serialString = "TEST! ";
const int BUFFER_LENGTH = 4;
const int DISPLAY_LENGTH = 30;
const int DISPLAY_HEIGHT = 7;
byte* datarow = new byte[DISPLAY_HEIGHT][BUFFER_LENGTH];
void setup() {
  for (int p = 0; p <= 12; p++) {
    pinMode(p, OUTPUT);
    digitalWrite(p, LOW);
  }
  SPI.begin();
  SPI.setDataMode(SPI_MODE2);
  SPI.setBitOrder(LSBFIRST);
  SPI.setClockDivider(SPI_CLOCK_DIV128);

  for (int row = 0; row < DISPLAY_HEIGHT; row++) 
  {
    for (int col = 0; col < DISPLAY_LENGTH; col++) {
      datarow[row][((col + ((BUFFER_LENGTH * 8) - DISPLAY_LENGTH)) / 8)] |= 
        ((font[serialString[col / 5]][col % 5] & (1 << ((DISPLAY_HEIGHT - 1) - row))) 
          ? 1 : 0) << ((col) % 8);
    }
  }
}

void loop() {
  for (int row = 0; row < DISPLAY_HEIGHT; row++) 
  {
    for (int col = 0; col < BUFFER_LENGTH; col++) SPI.transfer(datarow[row][col]);
    digitalWrite(row + 3, HIGH);
    delayMicroseconds(500);
    digitalWrite(row + 3, LOW);
  }
}

My text was static, so I now only built the buffer once. Would you believe it? The rendering was beautiful! I couldn't even notice a flicker. This was then short-lived when I realised I needed to scroll text. Again, there was no need to rebuild the buffer on each loop, so I wrote a procedure and only called it when there was text to move; defined by a delayed scroll rate.

This worked nicely, but there was an obvious jarring delay as the text shifted along. You could feel it re-processing in-between the character movements. I was still happy with it! I went ahead and reconfigured my initial display setup to use two Arduinos: one for the remocon+wifi and the other to receive serial data and light this dispalay.

But Wait, Why Don't I Build The Entire Text Buffer?

Whilst writing this up, it occurred to me that I was only rebuilding the buffer for the text that was visible on the display, not the entire string. If I had the full text buffer already created somewhere in memory, I then would only need to get SPI to send out the data from a specific bit offset. But, how do you start sending data via SPI from half-way through a byte? My fonts/characters were 5 bits wide and each byte had 1.5ish characters in them! I considered just shifting the pointer along the bytes, but the scrolling would be horrible.

So, I want to send out data from half way through a byte? Let's just borrow that table from above again:

Bytes 00 01 02 03
Pixels -- -- 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
Row 0 00 00 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 00 00 01 00 00 00 00 00 00 00
Row 1 00 00 00 00 01 00 00 01 00 00 00 00 01 00 00 00 00 00 00 01 00 00 00 00 01 00 00 00 00 00 00 00
Row 2 00 00 00 00 01 00 00 01 00 00 00 00 01 00 00 00 00 00 00 01 00 00 00 00 01 00 00 00 00 00 00 00
Row 3 00 00 00 00 01 00 00 01 01 01 01 01 01 01 01 01 01 00 00 01 00 00 00 00 01 00 00 00 00 00 00 00
Row 4 00 00 00 00 01 00 00 01 00 00 00 00 00 00 00 00 01 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00
Row 5 00 00 00 00 01 00 00 01 00 00 00 00 00 00 00 00 01 00 00 01 00 00 00 00 01 00 00 00 00 00 00 00
Row 6 00 00 00 00 01 00 00 01 01 01 01 01 01 01 01 01 01 00 00 01 00 00 00 00 01 00 00 00 00 00 00 00

Officially, on the 3rd loop of scrolling, I would want to send out the green bits. I'd also then want to send out five zeroes at the other end so that the scrolling text trails off and doesn't wrap around. I can't tell SPI to start at the 4th bit of a byte? So I'll need to create a new buffer and populate this as quickly as possible, line by line, just as we're about to render it. Thanks to the glory of the internet, there's always someone else who wants to do something similar. Effectively we have an array of bytes (unsigned chars, meaning we get all 8 bits) and we'll need to shift the first one and append the next one. It'll be very much like a catepillar crawling along.

So, if we build the full text buffer array when the string comes in, we can then push the relevant chunk into the display buffer. There'll be no font calculations, just bit for bit copying, and this will hopefully still be fast enough. We can even make sure we only do it line by line, and only as required... if we're on a byte boundary, then we don't need to shift anything!

I mentioned shifting above, and that's what we'll need to do to get the chunks of bytes into the final display buffer. If you look at the table above, you'll notice I coloured the cells in light green and dark green. The first 3 bits are light green and come from the 'end' of the first byte of the text buffer. The next 5 bits are from the 'start' of the second byte in the text buffer. This chain then continues on as we shift the bits. The >> and << C operators will assist us with this task as that's exactly what they're designed to do: shift bits in a variable in a certain direction by a certain amount. Let's try some pseudo-code first....

define a string buffer that has the string to display: "TEST!";
build a buffer for this string: stringBitBuffer[7][4];
define a display buffer that is enough for 1 panel: displayBuffer = byte[7][4];
if our scroll offset is a multiple of 8
  foreach stringBitBuffer[scrolloffset /8] SPI.writeRow();
  write extra zeroes at the end.
else if our scroll offset is NOT a multiple of 8
  foreach byte in stringBitBuffer starting from (scrollOffset/8)
    shift the first block of the text buffer right scrollOffset times
    make anything to the left of the text in that byte zero.
    move the resulting value into the current displayBuffer byte.
    get the next text buffer byte. 
    take the first (8 - scrollOffset) bits and add them to the current displayBuffer byte
  next
end if

Here's an animation showing what bits we want where...

Bytes 00 01 02 03
Pixels -- -- 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
Row 0 00 00 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 00 00 01 00 00 00 00 00 00 00
Row 1 00 00 00 00 01 00 00 01 00 00 00 00 01 00 00 00 00 00 00 01 00 00 00 00 01 00 00 00 00 00 00 00
Row 2 00 00 00 00 01 00 00 01 00 00 00 00 01 00 00 00 00 00 00 01 00 00 00 00 01 00 00 00 00 00 00 00
Row 3 00 00 00 00 01 00 00 01 01 01 01 01 01 01 01 01 01 00 00 01 00 00 00 00 01 00 00 00 00 00 00 00
Row 4 00 00 00 00 01 00 00 01 00 00 00 00 00 00 00 00 01 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00
Row 5 00 00 00 00 01 00 00 01 00 00 00 00 00 00 00 00 01 00 00 01 00 00 00 00 01 00 00 00 00 00 00 00
Row 6 00 00 00 00 01 00 00 01 01 01 01 01 01 01 01 01 01 00 00 01 00 00 00 00 01 00 00 00 00 00 00 00
TB 00 00 00 00 00 00 00 00 01 01 01 01 01 01 01 01 02 02 02 02 02 02 02 02 03 03 03 03 03 03 03 03
Bytes 00 01 02 03

The table above shows the 8 frames that need to be copied over to the framebuffer between bytes. Once the data has shifted a whole byte to the left, then the drawing routine just needs to start from the next byte, there's no need to send out the initial bytes if they're offscreen. We need to copy the bits in the main area of the table into the buffer bytes listed on top. I've added a new row at the bottom which indicates the first bit/byte of the text buffer (TB). With the duplicated Byte row underneath, you can see how we'll need a part of one text buffer and the rest of the next to fill the gaps.

Back to that forum post I mentioned above, and the description I gave about bit shifting, we can build up our buffer. We need to, based on the scroll offset, take the first (8 - offset) bits from every byte and then the 'offset' bits from the following byte. i.e. if the offset was 3, we want the last 5 bits of byte 1 and the first three of byte 2. If we're on the last byte, then we just fill the end with zeroes. The AND masks for row 2 are shown below. Row 1 is nearly all bits set... so wouldn't be a good test case.

Bytes 00 01
Pixels 00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15
Row 0 00 00 01 01 01 01 01 01 01 01 01 01 01 01 01 01
Row 1 00 00 00 00 01 00 00 01 00 00 00 00 01 00 00 00
AND #1 00 00 00 01 01 01 01 01 00 00 00 00 00 00 00 00
AND #2 00 00 00 00 00 00 00 00 01 01 01 00 00 00 00 00

And then we just need to shift the first 5 bits to the left... as we're stored in MSB.

#include <SPI.h>
#include "myfont.h"

int ccol = 0;
byte datarow[7][60];
int displayLength = 120;
int bytesRequired = (displayLength + (displayLength % 8) + 8) / 8;
int totalPixels = bytesRequired * 8;
byte frameBuffer[7][16];

unsigned long last_time = millis();
unsigned long scroll_time = millis();
unsigned long scroll_pause = millis();
unsigned long next_text_time = millis();
int charOffset = 0, loop_count = 0, charcol = 0;
String displayString;

void buildTextBuffer() {
  bytesRequired = ((displayString.length() * 5) / 8) + ((displayString.length() * 5) % 8 == 0 ? 0 : 1);
  totalPixels = bytesRequired  * 8;
  for (int row = 0; row < 7; row++) 
  {
    for (int b = 0; b < bytesRequired; b++) datarow[row][b] = 0;
    for (int col = 0; col < totalPixels; col++) {
      ccol = col + 8;
      datarow[row][ccol / 8] |= 
        ((font[displayString[col / 5]][col % 5] & (1 << (6 - row))) 
          ? 1 : 0) << (ccol % 8);
    }
  }
}

void buildFrameBuffer(int offset = 0) {
  int tOffset = offset / 8;
  int cOffset = offset % 8;
  int bb = 0;
  for (int row = 0; row < 7; row++) 
  {
    for (int b = 0; b < 16; b++) {
      bb = b+tOffset;
      frameBuffer[row][b] = (datarow[row][bb] >> cOffset);
      if (cOffset > 0) {
        if (bb <= bytesRequired) {
          frameBuffer[row][b] |= (datarow[row][bb + 1] << (8 - cOffset));
        }
      }
    }
  }
}

void setup() {
  Serial.begin(9600);
        
  for (int p = 0; p <= 12; p++) {
    pinMode(p, OUTPUT);
    digitalWrite(p, LOW);
  }
  
  SPI.begin();
  SPI.setDataMode(SPI_MODE3);
  SPI.setBitOrder(LSBFIRST);
  SPI.setClockDivider(SPI_CLOCK_DIV64);

  displayString = "Warming up ... ";
  buildTextBuffer();
  buildFrameBuffer(0);
}

String newString = "";
void getNextIndex() {
  for (int i = 0; i < 7; i++)
    for (int x = 0; x < 60; x++)
      datarow[i][x] = 0;
      
  if (Serial.available()) {
    newString = Serial.readStringUntil('\n');
    if (!newString.equals(displayString)) {
      displayString = newString;
      next_text_time = millis();
    }
  } else {
    displayString = "";
  }
  if (displayString.length() > 0) buildTextBuffer();
  buildFrameBuffer(0);
}

void loop() {  
  for (int row = 0; row < 7; row++) 
  {
    for (int col = 0; col < 16; col++) SPI.transfer(frameBuffer[row][col]);
    digitalWrite(row + 3, HIGH);
    delayMicroseconds(150);
    digitalWrite(row + 3, LOW);
  }

  if ((millis() - scroll_pause > 2500)) {
    if ((displayString.length() > (displayLength / 5))) {
      if (millis() - scroll_time > 30) {
        charOffset++;
        if ((charOffset / 5) > displayString.length()) {
          charOffset = 0;
          loop_count++;
          scroll_pause = millis();
          if (millis() - next_text_time > 5000) getNextIndex();
        }
        scroll_time = millis();
        buildFrameBuffer(charOffset);
      }
    } else {
      loop_count++;
      if (millis() - next_text_time > 5000) {
        getNextIndex();
        loop_count = 0;
      }
      charOffset = 0;
    }
  }
}

And it all actually worked... beautiful scrolling.

Further Possible Optimisations

We're sending the text into this Arduino from another Arduino over SoftSerial. Why send text when I could send the framebuffer bits instead? I wonder if sending (string-length * 5 * 7 * 8) bits would be quicker than the initial string and calculations on the secondary Arduino's side? Should I test it? Maybe later...

Filed under: Arduino No Comments
24Oct/190

Arduino Uno WiFi + TV Remote + LED Sign

Over the weekend, whilst watching TV, I realised that I needed some kind of sign to work out when trains were passing my apartment. There's a railway line right next to me and around four-or-more services pass daily. I'd prefer to be notified in advance and, since I have data sources to tell me when trains are about to come by, I want to make the most of it. Currently that's done by RSS both on my phone and PC, but the whole point is that I'm watching TV and don't want to have to be checking my phone every 2 seconds.

After getting the red LED marquee to work on the old ISA I/O card, I decided it was time to get it plugged into the Arduino. I actually already have an Arduino Uno Wifi next to my TV, which I've recently been using to translate my TV remote control codes (similar to what I did with the old B&O TV) into Yamaha remote control codes for control of volume with one remote. Adding the LED marquee would let me know when a button was successfully pressed ... and it would also tell me when trains were coming past!

I hadn't actually worked with the WiFi on the Arduino yet, so that was a bit of an experience. The RF side of things was working fine and I had already worked out the wiring requirements for the LED display thanks to the old I/O card.

Getting the ESP8266 Online

The Arduino Uno Wifi is an Arduino Uno board with an embedded ESP8299 microcontroller. The basic idea is that the two units talk serial to each other, when configured to do so. If you want to program one, or the other, you have to adjust the serial communications path on the board via a tiny row of switches. Jaycar has a minimal instruction sheet here, but it's enough to get you started. From the information, please set your Arduino Uno Wifi's tiny switch row to: 1:OFF,2:OFF,3:OFF,4:OFF - 5:ON,6:ON,7:ON - 8:OFF.

I would want this unit to act by itself as both a web server and web requestor, so I needed to find code for both. Before this we need to set up the Arduino IDE for use with the ESP8266. I found a great tutorial here, but otherwise, the steps are pretty simple. Open the Arduino IDE, go to Preferences and then paste this URL into to the additional boards list: http://arduino.esp8266.com/stable/package_esp8266com_index.json.

Screen Shot 2019-10-20 at 11.37.10 am

Once done, go to the board manager and search for ESP8266...

Screen Shot 2019-10-20 at 11.37.54 am

Install the relevant row and then restart the Arduino IDE. From here, you should have a new selection list in the boards drop-down...

Screen Shot 2019-10-20 at 11.39.41 am

...and then, once selected, a hideous amount of new options to adjust the programming!

Screen Shot 2019-10-20 at 11.44.27 am

The main advice is to not touch any of them, apart from the serial port. You'll need to work out which one your Arduino is connected to, but depending on if you even had serial ports in the first place, it should be pretty obvious. I'm working on a Mac Mini and my Arduino was connected via cu.usbserial-1460. From here, open the Serial Monitor and hit reset on the Arduino.

Screen Shot 2019-10-20 at 11.44.50 am

What garbage is that? I then started from the slowest and tested each baud... turns out if you set to 74880 then you'll get the following...

Screen Shot 2019-10-20 at 11.46.29 am

What a weird port number? It seems that just the initial header information is at that BAUD rate. Afterwards, it'll be at whatever you've set in the code you upload.

Running A Web Server

So, where were we? We have a connected device, so let's grab the first example from here and compile it. After installing the ESP8266 libraries, this code should compile with zero issues. If you happen to have any, then write a comment below, providing as much detail as possible. If it's worked, then check the code and adjust the Wifi SSID and password to match your local AP. Hit upload and cross your fingers and toes... hopefully you'll get the following output:

Screen Shot 2019-10-20 at 11.49.07 am

Yes! Now, you need to switch OFF switch 7 on the little switch header on the Arduino. This pin enables the programming mode and we now want it in the run mode. Note also that we've now set the baud to 115200, so change your serial monitor to that too... Do you get the following after hitting the reset key on the Arduino?

Screen Shot 2019-10-20 at 11.50.08 am

Associated! And with a nice IP! So what does the web browser say?

Screen Shot 2019-10-20 at 11.54.58 am

WIN!

HTTPS Web Requests with the ESP8266

The basic ESP8266 examples use WiFiClient, but this only works with HTTP requests. My server runs over HTTPS, so we need to use WifiSecureClient for secure web requests. To call an HTTPS service, you'll need to get the fingerprint from the SSL certificate. Browse to the site in a regular web browser and check out the properties of the lock next to the URL in the address bar. Somewhere in there you'll find the required data.

Screen Shot 2019-10-20 at 12.39.49 pm

And from there, edit the WifiClient Example to carry out an SSL request using these instructions:

#include <esp8266wifi .h>

const char* ssid = "********";
const char* password = "********";
const char* host = "api.github.com";
const char* fingerprint = "CF 05 98 89 CA FF 8E D8 5E 5C E0 C2 E4 F7 E6 C3 C7 50 DD 5C";

void setup()
{
  Serial.begin(115200);
  Serial.println();

  Serial.printf("Connecting to %s ", ssid);
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED)
  {
    delay(500);
    Serial.print(".");
  }
  Serial.println(" connected");
}


void loop()
{
  WiFiClientSecure client;
  Serial.print("[connecting to ");
  Serial.println(host);
  if (client.connect(host, 443)) {
    Serial.println("connected]");
    if (client.verify(fingerprint, host)) {
      Serial.println("[certificate matches]");
    } else {
      Serial.println("[certificate doesn't match]");
    }
    Serial.println("[Sending a request]");
    client.print(String("GET /") + " HTTP/1.1\r\n" +
                 "Host: " + host + "\r\n" +
                 "Connection: close\r\n" +
                 "\r\n");
    Serial.println("[Response:]");
    while (client.connected() || client.available())
    {
      if (client.available())
      {
        String line = client.readStringUntil('\n');
        Serial.println(line);
      }
    }
    client.stop();
    Serial.println("\n[Disconnected]");
  }
  else
  {
    Serial.println("connection failed!]");
    client.stop();
  }
  delay(5000);
}

And hopefully, you'll get something similar to the following in your serial terminal.

{ll⸮⸮|⸮d⸮|⸮d⸮b|⸮⸮⸮⸮s⸮b⸮c⸮⸮'o⸮dg'⸮⸮⸮cp⸮⸮dsdrlx⸮o⸮⸮d⸮⸮co⸮|l⸮⸮c⸮⸮go⸮d⸮⸮$`⸮'o$`gs⸮⸮⸮ob⸮$;⸮⸮gc⸮l⸮⸮dl⸮⸮d`⸮⸮'⸮

Connecting ...
scandone
........scandone
.scandone
state: 0 -> 2 (b0)
state: 2 -> 3 (0)
state: 3 -> 5 (10)
add 0
aid 1
cnt 

connected with COMEGETSOME, channel 6
dhcp client start...
ip:192.168.1.121,mask:255.255.255.0,gw:192.168.1.1

Connected to COMEGETSOME
IP address:	192.168.1.121
mDNS responder started

connecting to api.github.com
Using fingerprint '59 74 61 88 13 CA 12 34 15 4D 11 0A C1 7F E6 67 07 69 42 F5'
requesting URL: /repos/esp8266/Arduino/commits/master/status
request sent
headers received
esp8266/Arduino CI has failed
reply was:
==========
{"state":"pending","statuses":[],"sha":"1e17dddd895883806c9bfd3dd7b7042aa813d94f","total_count":0,"repository":{"id":32969220,"node_id":"MDEwOlJlcG9zaXRvcnkzMjk2OTIyMA==","name":"Arduino..."}
==========
closing connection

Nice, we can successfully pull data from a HTTPS source. Let's rig up a polling system to get the data each minute.

Polling for Data + Extra Buttons

So, we're able to pull data from the internet, but I also want this unit to be a mini webserver. If I used Delay() to spread out the web-polling calls, then I'd halt the entire unit. We can't have it wait when it needs to be handling button presses from the web interface. Instead, we'll take a snapshot of the current internal millisecond counter using millis() and then check how many milliseconds have passed to work out if we should try and grab new data.

#include <ESP8266WiFi.h>
#include <WiFiClient.h>
#include <ESP8266WiFiMulti.h> 
#include <ESP8266mDNS.h>
#include <ESP8266WebServer.h>   // Include the WebServer library

ESP8266WiFiMulti wifiMulti;     // Create an instance of the ESP8266WiFiMulti class, called 'wifiMulti'
ESP8266WebServer server(80);    // Create a webserver object that listens for HTTP request on port 80

void handleRoot();              // function prototypes for HTTP handlers
void handleNotFound();
void handleRemote();
unsigned long lastMillis;
const char* host = "trainfinder.otenko.com";
const int httpsPort = 443;
const char fingerprint[] PROGMEM = "BC 9C 56 D5 8E 7A FE CC 0C F7 A2 21 1F C5 7B 9A C0 FA 15 41";
String last_line = "";

void setup(void) {
  Serial.begin(9600);
  delay(100);
  Serial.println("\n!START");
  wifiMulti.addAP("COMEGETSOME", "blahblahblah");
  while (wifiMulti.run() != WL_CONNECTED) { delay(250); }
  Serial.println(WiFi.SSID());
  Serial.println(WiFi.localIP());

  server.on("/", HTTP_GET, handleRoot);
  server.on("/buttons", HTTP_POST, handleRemote);
  server.onNotFound(handleNotFound);
  server.begin();
  
  lastMillis = millis();
}

void loop(void){
  server.handleClient();
  if ((millis() - lastMillis) > (60 * 1000)) {
    WiFiClientSecure client;
    client.setFingerprint(fingerprint);
  
    if (!client.connect(host, httpsPort)) {
      Serial.println("!FAILED");
    } else {
      String url = "/some/cool/url";    
      client.print(String("GET ") + url + " HTTP/1.1\r\nHost: " + host + "\r\nConnection: close\r\n\r\n");
      while (client.connected()) {
        String line = client.readStringUntil('\n');
        if (line == "\r") break;
      }
      bool foundLastLine = false;
      String line = "";
      while(client.available()) {   
        line = client.readStringUntil('\n');
        if (last_line.equals("") || foundLastLine) {
          Serial.println(line);
          last_line = String(line);
          delay(6500); //space out the data to the Arduino
        }
        if (last_line.equals(line)) {
          foundLastLine = true;
        }
      }
      if (!foundLastLine) last_line = line;
    }
    lastMillis = millis();
  }
}

void handleRoot() {
  server.send(200, "text/html", "<form action='/buttons' method='POST'>" \
      "<input type='submit' name='pwr_plz' value='PWR!'/>" \
      "<input type='submit' name='vol_dn' value='VOL-'/>" \
      "<input type='submit' name='vol_up' value='VOL+'/>" \ 
    "</form><br/><br/>Last Line: <pre>" + last_line + "</pre>");   
}

void handleRemote() {
  if (server.hasArg("vol_up")) {
    Serial.println("!VOLUP");
  } else if (server.hasArg("vol_dn")) {
    Serial.println("!VOLDN");
  } else if (server.hasArg("pwr_plz")) {
    Serial.println("!POWER");
  }
  server.sendHeader("Location","/");
  server.send(303);
}

void handleNotFound(){
  server.send(404, "text/plain", "404: Not found");
}

Just for fun, here's the first shot of me writing the code. Instead of 'text/html', I was still outputting 'text/plain' and, well, as expected, got the following...

Screen Shot 2019-10-20 at 12.04.08 pm

After fixing that, the screenshot below was taken. Of course, it doesn't match the code above, but you get the idea.

Screen Shot 2019-10-20 at 12.06.02 pm

The basic idea is to print out the raw html to the browser. From here there's a form that has the buttons which post back to the ESP8266. The controller then sees the post, sends the command via Serial to the Arduino and then redirects the user back to the main form.

Screen Shot 2019-10-20 at 12.06.05 pm

Now to get the Arduino to consume it.

ESP8266 to Arduino

We've currently had the ESP8266 piped directly out the USB Serial port. Now we want it to talk to the Arduino internally. To do this, the tiny onboard switch needs to have all switches off except for 1 and 2. This then routes the RX/TX directly from the ESP8266 to the Arduino. From here though, the unit becomes a black-box. We have no idea what the ESP8266 is saying to the Arduino, and vice-versa, as the connection via the USB serial port has been severed. We can only hope that the data we're sending through is what we want!

If you check the shots above of the data the ESP8266 sends, you can see that we want to skip a bit of the earlier stuff. To do this, I defined a "!START" command which the Arduino should look for before bothering to deal with any of the data from the serial channel. From there, anything starting with a "!" is a command, whereas anything else is to be printed to the LED Display.

Oh yeah, that TV Remote bit

I previously had an amplifier that ran over ARC over HDMI and also received controls... but half the time it'd switch on to Optical and just would not switch back to ARC without a lot of screwing around. No amount of power cycling, cable switching or button pressing would get it back. So it went to the farm. Instead, I purchased a beautiful old Yamaha amp for AUD$5 and then realised I had to get up and control it by hand. Not optimal, so I search google for the remote codes.... they didn't exist, so I search eBay for a spare remote... they didn't exist... but one guy in Spain had a company that programmed third-party remotes and had one for this amp!

It arrived, and I plugged it through the same code I used for the B&O TV. Over the serial, the codes and bits were reported, showing me that it used the "NEC" protocol. I recorded the codes for the buttons I wanted. I then did the same with my Sony TV Remote for buttons that I wanted to re-purpose. The goal was to have the TV Volume buttons also trigger the amp volume. This worked nicely, via an external IR LED that sits in front of the amp.

Of course, the TV volume still shifts, so I made the amp volume shift 4 times per button press... meaning the surround is always louder than the TV. Doesn't hurt to have a little bit of tinny sound from the TV also... a second center channel?

To make everything compile, install Ken Shirrif's IRremote library, available via the Arduino GUI...

Screen Shot 2019-10-20 at 2.31.18 pm

The code is in the next segment.

Hooking up the LED Sign

I've used a font found online, but it seems that Arduino has one built-in? 5x7 font. Either way, the 3 data lines and 7 row-enable pins need to find homes in the Arduino digital IO pins. Make sure you don't use digital pin 0 or 1 as that's the serial channel to the ESP8266. I found this out later below...

Once hooked up, just use a similar loop process as per any LED display: Disable all rows, send out the columns to be lit for the first row, enable that row and quickly disable it. Then do the same for the next 6 rows. As quickly as possible! More information on lighting this sign is over here.

#include <IRremote.h>
#include "myfont.h"
#include "digitalWriteFast.h"

int RECV_PIN = 13;
IRrecv irrecv(RECV_PIN);
decode_results results;
IRsend irsend;

void setup()
{
  Serial.begin(9600);
  irrecv.enableIRIn(); // Start the receiver

  pinModeFast(0, OUTPUT);
  digitalWriteFast(0, LOW);
  pinModeFast(2, OUTPUT);
  digitalWriteFast(2, LOW);
  pinModeFast(12, OUTPUT);
  digitalWriteFast(12, LOW);

//matrix led row drivers
  for (int p = 0; p <= 11; p++) {
    pinModeFast(p, OUTPUT);
    digitalWriteFast(p, LOW);
  }
}

unsigned long last_time = millis();
unsigned long scroll_time = millis();
unsigned long scroll_pause = millis();
bool is_off = true;
int loop_count = 0;
String serialString = " Warming up..";
bool found_start = false;
int charOffset = 0;
int ccol;

void loop() {
  if (irrecv.decode(&results)) {    
    loop_count = 0;
    switch (results.value) {
      case 4841:
      case 0xA90:
        irsend.sendNEC(0xDE21807F, 32);
        if (is_off) {
          serialString = " Power ON!";
          is_off = false;
        } else {
          serialString = " GOOD BYE!";
          is_off = true;
        }
        break;
      case 0x490:
        irsend.sendNEC(0xDE21B04F, 32);
        delay(50);
        irsend.sendNEC(0xDE21B04F, 32);
        delay(50);
        irsend.sendNEC(0xDE21B04F, 32);
        serialString = " VOLUME UP!";
        break;
      case 0xC90:
        irsend.sendNEC(0xDE21708F, 32);
        delay(50);
        irsend.sendNEC(0xDE21708F, 32);
        delay(50);
        irsend.sendNEC(0xDE21708F, 32);
        serialString = " VOLUME DOWN!";
        break;
      case 0xFFFFFFFF:
        irsend.sendNEC(0xFFFFFFFF, 32);
        break;
        //sony teletext red
      case 21225:
        irsend.sendNEC(3726721215, 32);
        serialString = " INPUT!";
        //sony teletext green
      case 13033:
        irsend.sendNEC(3726717135, 32);
        serialString = " MODE UP!";
        break;
        //sony teletext yellow
      case 29417:
        irsend.sendNEC(3726735495, 32);
        serialString = " MODE DOWN!";
        break;
      default:
        Serial.println(results.value);
        Serial.println(results.bits);
        break;
    }
    delay(250);
    irrecv.enableIRIn();
    irrecv.resume(); // Receive the next value
  }

  if (charOffset == 0 && Serial.available() && ((millis() - last_time) > 4000)) {
    last_time = millis();
    serialString = Serial.readStringUntil('\n') + '\0';
    loop_count = 0;
    scroll_pause = millis();
  }
  
  //draw LEDs
  for (int row = 0; row < 7; row++) 
  {
    for (int col = 0; col < 90; col++) {
      if (((col / 5) + charOffset) < 60) {
        ccol = (charOffset * 5) + col;
        digitalWriteFast(2, ((font[serialString[ccol / 5]][ccol % 5] & (1 << (6 - row))) > 0) ? HIGH : LOW);
      } else {
        digitalWriteFast(2, LOW);
      }
      digitalWriteFast(12, HIGH);
      delayMicroseconds(5);
      digitalWriteFast(12, LOW);
    }
    digitalWriteFast(row + 5, HIGH);
    delayMicroseconds(75);
    digitalWriteFast(row + 5, LOW);
  }
  
  if ((millis() - scroll_pause > 2500)) {
    if ((serialString.length() > 18)) {
      if (millis() - scroll_time > 180) {
        charOffset++;
        if (charOffset > serialString.length()) {
          charOffset = 0;
          loop_count++;
          scroll_pause = millis();
          if (loop_count > 10) serialString = "";
        }
        scroll_time = millis();
      }
    } else {
      loop_count++;
      if (loop_count > 100) serialString = "";
      charOffset = 0;
    }
  }
}

Overloading Arduino Pins

Anyone watching above, or in the previous post where I hooked the display up to the old PC, was probably screaming in their seats. It's well known that you should NOT draw too much current through the pins on any microprocessor. As soon as I started hooking up more modules onto this unit, things started going haywire and it quickly dawned on me. My seven line-enable pins were being brought LOW to enable the rows... but these were actually working as the single GND conduit for the entire bloody LED display!

DSC01969

I quickly referred to the previous module that ran this sign and saw there was a 74LS145 binary counter in between the Atmel Microcontroller and the 7 enable rows. A quick review of the datasheet shows that this chip can 'sink' up to 15v and is 'great for driving LEDs'. Right... they were properly routing the GND wires away from the microcontroller and using the 145 as a set of transistors. They even saved 3 data lines in the process! I only had a ULN2003 (which would still use 7 lines) and there were no 74LS145 to be found at Jaycar!

DSC01988

This now directed the current away from the microcontroller... but didn't enhance the brightness.

Speeding up the Refresh Rate

It occurred to me that my refresh rates weren't anywhere near as good as the original controller that the sign contained. My loop is busy checking clocks, serial ports and mucking around, so it was never going to actually go fast enough. The code above could run a single segment OK, but started to scale worse as the bit shifting got longer. One trick, which is actually already implemented in the source above is to use an external library to make the digital pin manipulation faster.

With this implemented, there was less flickering and I found that three segments (90 LEDs per row) looked good. One final note is that you shouldn't really use pins 0 or 1 if you've got the serial enabled. They actually present the TX/RX lines and the serial data and the serial data to the Arduino is visible on them. In this case, it was the serial data to/from the ESP8266. In the case below, I had the three data lines required to drive the display on digital pins 0,1,2...

DSC01980

Shifting those to spare pins higher...

DSC01983

Yessssss! It works really nicely.

Filed under: Arduino No Comments
8Jun/180

Arduino – How to save power (and control 5v+)

For the first time ever, I get to play with Arduino at work! We're doing a little bit of work on tracking and the goal is to acquire as much data as possible and report it to a webserver. Therefore I've got sensors, GPS and GSM. As one can imagine, this consumes a lot of power, and as one can also imagine: the goal will be to obtain as long of battery life as possible.

Each of the modules requires either 3.3v or 5v and these are usually hardwired into the power supply. They also require anywhere from 50mA to 2A (GSM when sending data or searching for signals) and therefore can't be controlled directly from pins on the Arduino. Due to this I'll explain a proper mechanism to switch their power supplies on and off. Also, we'll then want to dig into power saving on the Arduino itself, so stick around.

Controlling modules from Arduino digital pins

Digital pins on the Arduino are only good for 30mA maximum. In fact, you shouldn't even be going anywhere near this. Hence, using one to provide +5v or GND to a module will cause that module's current needs to flow through the pin. For example, the GPS module I bought from Jaycar needs anywhere from 0-70mA. Although low, this is still too high for a digital pin.

The goal therefore is to use an electronic switch. You might be thinking 'relay', but you'll also need to note that they require current which can peak above the digital pin tolerance! You'll find that relay modules don't connect the relay actuator to the input.

arduino-5v-control

The answer here is to use a transistor to allow current to flow to your modules. Depending on your current requirement, I can either recommend a 2N2222 (for anything up to 600mA) or a TIP31 (good for 3A.) I also want to switch my GSM module, and that needs 2A, so I bought a handful of TIP31s.

Using them is very straight-forward. Connect your VCCs up to whatever power source is needed and embed the transistors in the ground path. From here, switching the digital pin high will let current flow and complete the circuit for the module. Make sure you set your digital pinMode to OUTPUT.

Ideas for saving power via code

The basic idea is to put the Arduino to sleep whenever there's nothing to do. My device only needed to report every 15 minutes, so in the meantime I tried to make it snooze. It turns out that, based on internal timers and interrupts, you can only get a maximum of 8 seconds power down out-of-the-box. This wasn't quite the 15 minutes I was looking for, but numerous sources online said to just keep putting the unit to sleep for 8 seconds over-and-over.

There's some great reading here by Nick Gammon on power saving. Anything from lower frequencies to shutting down certain parts of your project. You'll also find a blog post here at the Arduino Playground describing other options. Just make sure you read that entire second article as other contributors have made corrections or other important points. Tis great to see the community involved.

...I'll come back to power saving once the project gets started up again. The proof-of-concept worked and now we need to convince the powers-that-be to let us continue.

Filed under: Arduino No Comments
7Jun/180

Arduino – Don’t buy the SIM900A for Australia!

tl;dr = This is a 2G-only device. There's only one network left, which shuts down on the 30th of June, 2018!

...but I needed a connection for work and I bought this unit from a seller on eBay without researching... so... compounding the error, I then also bought a SIM card for the network that'll still work: Vodafone. Thanks to Happymacer on Instructables who posted an article discussing how to use this device in Oz!

Note that I'd previously tried to use both Optus and Telstra SIM cards and got the following output when running the COPS command.

AT+COPS=?
+COPS: (3,"VODAFONE","voda AU","50503"),,(0-4),(0-2)
OK

Yey! Vodafone! But I'm on Telstra/Optus? So I can see a Vodafone tower, but it's prepended with 3 which actually means I'm not allowed to use it. Sorta makes sense as Optus and Telstra switched off their 2G networks ages ago and Vodafone must have decided that lingering devices on other companies' plans aren't allowed to use their towers.

I bought a Vodafone SIM and ran the commands again...

AT+COPS=?
+COPS: (2,"VODAFONE","voda AU","50503"),,(0-4),(0-2)
OK

Woweee! It's there. 2 means Roaming... which is a little weird, but it must be trying to hint to my device that 2G isn't the best method and that there should be a 3G tower around somewhere. Of course, even if there was, we couldn't connect to it. So I forced the connection with the following command...

AT+COPS=1,2,"50503"
+CREG: 2
+CREG: 1
OK

Those CREGs afterwards were output because I had previously run AT+CREG=1. This command tells the device to report any changes to the carrier connection. What you can see is it connecting to the tower in roaming mode (aka 2) and then connecting correctly in local mode (aka 1.) We're on!

Testing SMS

If you wanna test an SMS, then you can do so pretty easily. Rig up a SoftwareSerial example in Arduino and make sure you can send commands to the SIM900A via the Serial Monitor. From here, you can use the following script to send an SMS.

AT+CMGF=1
OK
AT+CMGS="+61xxxxxxxxx"
>

Note above ... you'll get the > prompt to enter the message. Once you're done, the unit requires a CTRL-Z control character to realise it's the end of the message. To do this, you'll need to open your favourite text editor, type ALT+026 (hold down ALT and type 0 then 2 then 6) and then copy the resulting character. It will be a black SUB if you use Notepad++. Now paste this into the text entry field in the Serial Monitor and hit enter.

Sending Data

From here, Jens Christoffersen at All About Circuits has helped us greatly with his article: Using a SIM900A to Send Sensor Data to a Website. Scroll down far enough and you'll find his code to send data to a server. He uses printf statements, as he's using a PIC and talking direct to the serial port... we'll have to use SoftwareSerial and print(ln).

AT+CGATT=1
OK
AT+SAPBR=1,1
OK
AT+SAPBR=2,1
+SAPBR: 1,1,"100.84.75.204"
OK
AT+HTTPINIT
OK
AT+HTTPPARA="URL","http://www.website.com/index.php?data1=1&data2=4"
OK
AT+HTTPPARA="CID",1
OK
AT+HTTPACTION=0
OK
+HTTPACTION: 0,200,0
AT+HTTPTERM
OK

Above is an example chat with the unit to send data. Paste each AT line into the Serial Monitor first to make sure you get the expected responses. The URL above is rubbish, so put one in that'll work properly instead. The 200 in the HTTPACTION response is the happy HTTP code.

Filed under: Arduino No Comments
21Aug/1710

MSX – Arduino as Tape Drive (CASDuino)

As usual, no vintage equipment is complete until you've maxxed it out with all possible peripherals! I had previously made an old 'Datasette' drive work for a Commodore 64 and so started searching for an appropriate tape player for my MSX. It turns out that you either pay big dollars for a specific data recorder, or you use an off-the-shelf tape player and hope-it-works.

I then stumbled across Arduitape (aka TZXDuino). The slogan says it all: 'ARDUITAPE MARK II - THE TAPE PLAYER REPLACEMENT SD CARD SYTEM FOR 8-BIT COMPUTERS'. After a lot of digging on the blog, the instructions presented themselves. As you can see, the instructions aren't as clear as they could be and so I henceforth present the complete construction and usage of an Arduino as a fake Tape Recorder for an MSX.

Update: Current link to TZDuino 1.8 is here. Seems the links above are now dead :(

Ingredients

I ended up testing out multiple components during this build. I initially started with a 128x128 LCD but found that the libraries required to run it used too much memory and therefore the whole project was useless on a UNO/Leonardo. Instead I switched back to a 16x2 LCD.

Component Substitute Comments
Arduino Nano Arduino Leonardo r3 This Leonardo r3 from Jaycar worked fine, or a UNO.
16x2 Character LCD Find any 16x2 I2C LCD from eBay.
SD Card Shield SD Card Module Jaycar also has a full shield for SD Card reading, but we don't need that much infrastructure.
AMP Shield Arduino Compatible 2 X 3W Amplifier Module Different, but with two channels, we can use one for input.
4 x 4.7k resistors Filter Board
3 x 4.7nF Capacitors Filter Board
1 x 100nF Capacitor Filter Board
2 x 3.5 mm Female Jack PS0122 (One is for recording... can we get it to work?)
1 x 2.5 mm Female Jack PS0105
5 pushbuttons SP0711
Some kind of box to put it all in.

From here, I'll describe how to hook up and test each component to make sure that you build up a stable base for troubleshooting!

The Circuit

Here's an overview of what we're building. It's really just a rigging of off-the-shelf components, apart from the filter board.

CASDUINO

Note that the buttons aren't in the exact order. You can customise which button does what below.

Arduino

I used both a Uno and a Leonardo whilst constructing this. I bought the Leonardo as I thought it had more RAM than the Uno. Turns out it doesn't and so I switched from the 128x128 memory-expensive LCD to a simple 16x2 LCD. Either way, grab an Arduino and a nice case to house it in.

DSC00043

DSC00047 DSC00049 DSC00050

Punch holes where required and mount it all in place with hot glue.

LCD

This was a quick solder and plug-in. VCC and GND to the Arduino. SDA and SDL to analog pins A4 and A5. Make sure you have the daughterboard on the correct way around. It's on backwards in the first picture below. In the second and third pictures you'll see that you can't see the daughterboard as it's aligned behind the LCD.

DSC00118 DSC00119 DSC00121

If you get a single bar of black blocks, then chances are you have the I2C daughterboard on backwards. There's something that looks like a 'pin 1' designation on the board, but this only worked once I plugged it onto pin 16! I have it on BACKWARDS on the first shot above!.

SD Card

This is another I2C device which means it just needs to be wired into the bus. Again, hook VCC and GND to the Arduino. Then hook up CS to D10, SCK to D13, MOSI to D11 and finally MISO to D12.

DSC00070 DSC00067 DSC00069

You now should get a new message showing the first file/folder in the root directory. Go test out your google-fu to find CAS files for the MSX. You'll need one to test with.

DSC00122

You'll want to start showing-all-files-and-folders in Windows to get rid of the hidden items that'll now show up on this device. There's no filter in the card to disregard the kludge that OS' keep hidden on disks.

Buttons

These are easy enough... they just need a common ground and then 5 wires to the specified digital inputs. You can customise the order of your buttons, but in the end make sure you have then connected to the associated inputs of btnPlay, btnStop, btnUp, btnDown and btnMselect.

DSC00124 DSC00128 DSC00130

Not a TZXTape? Come again? Oh right, we're meant to be using CASDuino, not TZXDuino!

Filter and Amp

UPDATE: You can skip this circuit... just wire the the audio out straight into the amp board.

This little board is pretty straight-forward. I built it up as per the instructions.

DSC00139

You then need to provide GND and Audio In (Digital Pin 9) from the Arduino. Audio Out is fed into your Amplifier, which happens to be R-In on my tiny board from Jaycar. Yes, I'm using a RED wire for GND on the filter board, running to GND on the button row. It's a really good idea to tie ALL GNDs together wherever possible.

DSC00140

Disregard my colour-coding... that blue is actually GND and is using the GND rail from the LCD panel. Black is audio-out from the filter board to Audio-in on the amp. The amp then also needed VCC and all GND pins joined. From there, add on the 3.5mm headphone socket.

DSC00142

At this point I actually plugged the output into my stereo. An awful noise, to the tune of the data loading of the Commodore 64 (or even a modem dialing up), played loudly! Data!

DSC00145

Finally hook up the little 2.5mm socket to GND and D6. This will allow the MSX to tell the player when to play/pause.

Loading a game

With everything hooked up, I turned the device on. I then powered up the MSX, with no cartridges installed. At the BASIC prompt, my CASDuino started flickering between play/pause. It looked like the remote-control signal was floating instead of being pulled high or low. Regardless, I typed in the magic command: RUN"CAS:" (yes, double-quotes and all)

The CASDuino settled on PLAYING and I heard interference through the TV Audio!

DSC00147

Found:TURTLE appeared... but then it crashed?

DSC00149

Turns out you can set the BAUD rate of the tape playback. Default is 3600, but this was too high for my construction skills, or maybe even my MSX.

DSC00162

Setting this to 1200 or 2400 saw the game (slowly) load!

Teenage Mutant Hero Turtles

Did you know that UK/Ireland preferred Heroes over Ninjas? Supposedly Ninjas were too thought-provokingly violent. Either way, the game loaded. If you want to play with the keyboard, keys Q and A are UP/DOWN and keys O and P are LEFT/RIGHT.

DSC00151

DSC00153 DSC00154 DSC00158

DSC00160 DSC00156 DSC00157

Final Notes

Grab your glue gun and secure everything. This will hold it in place and also insulate any floating components.

DSC00164

DSC00166 DSC00167 DSC00168

Then mount the headphone plugs and close the box... it'll look much neater :) I ended up mounting a socket for the recording plug also... although it's not currently connected to anything. Might try and play with that in the future!

DSC00170

..and don't forget to clean up..

Filed under: Arduino, Retro 10 Comments
22Jun/160

Serial Train Controller (C64+Arduino)

I'd previously attempted to use the parallel port for train control but have now switched to the Serial port for communications. I've also slapped an Arduino in between the layout and the controlling (Terminal) device. The Arduino will be programmed to understand data coming in from the Serial port, which can be fed from any device which is capable of outputting an RS-232 signal. For the example below, I'll be doing this from a Commodore 64. See this article to learn how to build a Serial Port for your C64.

Designing a communication language

Since we'll be using the RS-232 standard, we're able to define a language that any Terminal can speak to control the Arduino. This will be based on ASCII for command verbs and then raw byte data where targets and values are required. For instance, we could set throttle number 1 to 150 or point number 4 to left. Without being too tricky, 3 bytes is adequate to communicate these commands: C|T|V. If we wanted the data messages to be 'human readable' (so you can print them straight out to a file or other serial line), then we should send through 7 bytes C|TTT|VVV, where the target and value are padded out with zeroes. Fortunately, we only need these two to be 8-bit values, which is the definition of a byte. We'll simply have to convert them to ASCII if we ever want to print or display them.

Next, we'll need a command separator. I've chosen the exclamation mark '!'. Hence, when sending commands, our messages will be 4 bytes long. The recipients will need to wait data, ensure there is at least a full 4-byte message with a terminator and then start processing.

Important Note: As is described down below, I found out the hard way that not all text is equal. I've naively mentioned 'ASCII' two paragraphs above and jumped straight onto the C64, compiling strings and sending them down the wire. For some reason, although I full-well knew that I was coding in PETSCII, I neglected to think that it would actually send it down the line! Long-story-short, when the C64 sends a capital A, it actually sends a character that does not map to ASCII. Also, when it sends a lower-case A, it is actually seen in ASCII as an upper-case A!

Moral of the story? Make sure you understand what character set each device is transmitting and receiving!

Talking back to the Terminal

Although everything written above indicates that most of the communication will be one-directional; this won't always be the case! Some of the commands in the table below will actually be asking for data from the layout. This will be in the form of sensor blocks, where optical or occupation sensors will exist on the layout and be wired to binary input data. The Arduino will be hooked up to shift registers which, when daisy-chained, will be capable of 'watching' up to 64 inputs.

To get this data back to the terminal, we'll send through a message of 10 bytes. This will start with the letter S and be followed by 8 bytes of binary data indicating the state of all connected sensors. Finally a '\0' will be appended to indicate the end of the message. The Arduino wont expect a response to this; if the Terminal has failed to receive, then it can simply request the data again.

The Command Table

Now that we have our expectation of 3 data bytes and one terminator per message, we can start to define all of the commands expect to send.

Command Target Data
T

Set Throttle

1 or 2

The Arduino will have two separate PWM throttles.

0 to 255

The throttles will supply 12v DC PWM. The wave-cycle will be dependent on the value from 0-255. This maps to 0-12v. Hence 128 should be roughly ~6v.

D

Set Direction

1 or 2

Each throttle has it's own direction.

F or R

Forward or Reverse.

P

Toggle Point

1 to 8

8 separate points will be connected.

L or R

We'll use ASCII here to make life easier. Left or Right can be specified.

S

Read Sensors

0

We don't need to specify a block... we'll get the whole 8 bytes back regardless.

0

No value required here. We'll send a zero for padding.

Code for the Arduino

The Arduino will need to keep an eye on the serial port and act on commands when they appear on the channel. Data from the serial port will come in as singular bytes, so they'll need to be written into a buffer and processed once a whole message is found. In case the Arduino struggles, it'll need to be able to understand what a whole message is and discard anything that looks incomplete.

I'll skip the bits on PWM throttles, reading sensors and LED Matrices here... I'll describe all that in another article. Currently you can find individual articles on each of these topics on this site if you need the information straight away.

#include <SoftwareSerial.h>
SoftwareSerial mySerial(8,9); // RX, TX

void setup() {
	// set the data rate for the SoftwareSerial port
	mySerial.begin(2400);
}

...

void processCommand() {
	int x = 0;
	switch (serial_buff[0])	{
		case 'T':
			setThrottle(serial_buff[1], serial_buff[2]);
			break;
		case 'D':
			changeDir(serial_buff[1], serial_buff[2]);
			break;
		case 'P':
			//adjust point
			break;	
		case 'S':
			//read sensors and return the data.
			break;
						
	}
	
	for (int x = 0; x < SERIAL_MAX; x++) serial_buff[x] = '\0';
	last_pos = 0;
}

...

void loop() {
	//if there's data and we've not read the end of the current message.
	if (mySerial.available() && (last_pos == 0 || last_char != '!')) {
		last_char = mySerial.read();
		serial_buff[last_pos] = last_char;
		last_pos++;
		if (last_pos > SERIAL_MAX) {
			//then we need to do something drastic
		}
		if (last_char == '!') processCommand();
	}
}

The snippet above checks if there's data and if we don't already have data. If the buffer is empty, then it'll read a character into it. If that character happens to be '\0' then it'll prevent further reading and process the message.

Controlling with a Commodore 64

cc65 has all the libraries we need to get the serial interface up and running; see more on that here. We'll use a text-based interface and control everything with the keyboard. At a later date I'll write a GEOS-based GUI.

I attempted to use the Tiny Graphics Interface libraries that cc65 provides. Unfortunately, that would've also meant writing a text renderer or graphical font library as the basic 'text out' for TGI isn't implemented on the C64. Staying with text-mode was to be easier and PETSCII has enough cool symbols to draw throttles and the like.

c64-trainctl2

The screen displays the throttle, just one for now, and the direction. It also provides a clock and a schedule. The user can add and edit items in the schedule and, when in run mode, these will be executed accordingly.

void sendCommand(unsigned char c, unsigned char t, unsigned char v) {
	ser_put(c);
	ser_put(t);
	ser_put(v);
	ser_put(33);
}

...

void main() {
	...
	sendCommand('d', 1, current_direction);
	sendCommand('t', 1, current_throttle);
	...
}

I've left out most of the guff ... I'll include the full source soon. For the meantime, you'll see that I've put individual characters to the serial channel for reading at the Ardunio end. Specifically they are lower-case! You'll also note that I write 33 instead of the literal character !. The reason is, of course, that the exclamation mark in PETSCII is not the same as ASCII.

What's Next?

This works. The train happily moves back and forth using the signals sent from the C64! It's overly boring though and based on the clock. I want sensors read back to be able to trigger events... so I'll hook up the multiplexing and post again shortly.

19Jun/160

Commodore 64: Serial Interface to Arduino

So, in my previous post, I was heading towards building an archaic circuit to control trains with the User Port. This would've worked, had I spent a lot more time and built a very complex circuit. Instead I've now chosen a new path... let's hook the C64 up to an Arduino and do most of the work there. The C64 can be the display and input/output for controlling the trains.

Interfacing both components

The C64 User Port has both a 'parallel port' with 8 i/o pins and a serial port. I initially wanted to use the parallel pins, but came to the conclusion that I'd have to write my own language on both sides and deal with the data transfer timings and clock synchronisation. Instead, it'll be easier to use industry-standard RS-232!

I suppose this is a bit of a cop-out. I wanted to build something that was dated to the level of technology that existed back when these machines were in their prime... unfortunately my electronic knowledge isn't up to scratch... so getting to a variable 12v output wasn't overly easy. It also would not have been PWM. Due to all this, including the Arduino into the mix isn't such a bad idea. Plus, everyone I'd asked for help told me to do this... even sending me links to my own blog posts :)

DTE vs. DCE

Serial plugs have a single channel, with each end having one transmit (TX) and one receive (RX) pin. Each end will send data down the cable via the TX pin and expect to receive data on the RX pin. Standard serial cables are 'straight through', meaning that the TX pin is connected to the TX pin and likewise with RX. Doesn't quite make sense, does it? How are to separate devices meant to eachother if they are both transmitting down the same singular TX wire and hearing silence on the RX?

This all becomes clear once you realise that devices fit into two categories: DTE (data terminal equipment) and DCE (data circuit-terminating equipment, also known as data communication equipment.) In the end, these two devices boil down to one being the master (the DTE) and one being the slave (the DCE.)

Of course, you can purchase 'cross-over' cables for serial connections. These are known as null-modem cables and allow you to hook two DTEs together. Say, for example, you want to transfer a file from your PC to your Amiga, or somesuch!

In my previous serial project, when I connected the IBM receipt printer to the Arduino, I needed the Arduino to be the master, and so I hacked around until I had correctly configured it as a DTE. This time around we want the Arduino to be the DCE. Due to this, be careful to follow the pinouts and wiring from the serial port to the MAX232 in the circuits below!

Note: For further reading/wiring on RS-232, there's a good article at avr Programmers and another at Advantech.

C64 Serial Port

The User Port on the C64 also has serial connections. These are TTL and so need to be brought up to the RS-232 5v standard. The MAX232 IC will do this for us quite easily. We'll also use one at the other end for the Arduino.

C64-RS232

The circuit is derived from 8bitfiles.net. This circuit was also correct in that the pins are wired up as DTE. This means that you could use it, as-is, to also hook to a modem or any other DCE device.

The MAX232 needs few extra components. Fortunately, these extra components are identical on both ends, so buy everything in duplicate! The capacitors are all 1.0uf electrolytic. I used 1k resisters on the LEDs so as not to draw too much current from the User Port.

Arduino Serial Port

This is nearly the same circuit as the C64 end. The funniest thing happened here... if you google enough 'arduino max232' you'll just loop back around to my post, from ages ago on interfacing an IBM printer to the Arduino. Just make sure you don't use that circuit! It's DTE, we need DCE as per below! I've left out the RTS/CTS as I don't intend on using any form of handshaking in this project. It's still in the circuit above for the C64 so that you can use the port for other purposes.

ARDUINO-RS232

As per usual, make sure you DO NOT apply 5v in the wrong direction... I did and it ruined a few caps and possibly the IC. Garbage then came through the serial port. If this ever happens, then throw out the components and start again; you'll never be able to trust them.

Also make sure that you use the 5v pin on the Arduino. AREF is NOT a valid voltage source.

Hooking it all together

Build both circuits above and give one a male and the other a female db-9 connector. The DCE device usually gets the female, so put this on the Arduino-side!

DSC03947 DSC03956 DSC03954

DSC03955 DSC03950

If you want to roll your own cable, then grab some grey IDC and two crimp-style plugs. Just make sure that you have pin 1 matched to pin 1. If you're splitting the cable, then paint a wire (or use a marker) to ensure that you get the orientation correct. It's really easy to confuse pin 1.

DSC03998 DSC04000 DSC04002

From above, you can see the pin numbering. I slid the second port all the way to the end, prior to crimping, to ensure that the numbers matched up. Using the red '#1 wire' on the cable worked wonders too.

Testing with Strike Terminal 2013 Final

Download Strike Term 2013 Final from here and then get it to your C64. I copied the D64 to my SD2IEC and loaded it up. Hit M and select User port. Hit b and switch it to 1200 Baud (or other baud, depending on what you've configured in the Arduino.)

DSC03976 DSC03978 DSC03957

DSC03958 DSC03964 DSC03965

Once ready, hit f5 and then hit enter on the default server. This'll start sending modem AT commands down the serial and they should start showing up at the other end. Either open the Arduino Serial Monitor... or edit the code to display it. I bought some 8x8 LED Matrices to render the data coming in.

DSC03963 DSC03971

There were no real caveats here... everything just worked! Press f3 to get to the terminal. Hit commodore+e for local echo and then commodore+i to 'send id'. You should now be able to type freely... everything will be sent down the wire.

DSC03983 DSC03984 DSC03985

DSC03986 DSC03987 DSC03988

At that point I only had one matrix... so the last char typed was displayed.

Writing C code to use the Serial Port

Nanoflite has written a 2400 baud User Port Serial Driver for cc65. I originally tried to use this at 1200 baud, as that's what I'd been using everywhere and heard it was the max the User Port was capable of. It turns out that this driver only supports 2400 baud! Download it and put the source somewhere.

Switch to the driver directory and compile it:

cl65 -t c64 --module -o c64-up2400.ser c64-up2400.s

Copy this to the folder that has your source file it. I slightly modified the example.

#include <stdlib.h>
#include <stdio.h>
#include <conio.h>
#include <serial.h>
#define DRIVERNAME  "c64-up2400.ser"

static const struct ser_params serialParams = {
    SER_BAUD_2400,      /* Baudrate */
    SER_BITS_8,         /* Number of data bits */
    SER_STOP_1,         /* Number of stop bits */
    SER_PAR_NONE,       /* Parity setting */
    SER_HS_NONE         /* Type of handshake to use */
};

int main (void)
{
  int xx;
  
  clrscr();
  puts("C64 serial ...");

  // Load driver
  ser_load_driver(DRIVERNAME);

  // Open port
  ser_open(&serialParams);

  // Enable serial
  ser_ioctl(1, NULL);

  for (xx = 0; xx < 10; xx++) {
	ser_put('C'); 
	ser_put('6'); 
	ser_put('4');
	ser_put('.');
	ser_put('.');
	ser_put('.');
  }
 
  return EXIT_SUCCESS;
}

Compile this:

cl65 -t c64 -O -o trainctl2 trainctl2.c

I then put it on the SD2IEC and loaded it via LOAD "0:TRAINCTL2",8 followed by a RUN.

DSC03993 DSC03989 DSC03991

DSC03992 DSC03994 DSC04003

Shit... worked... this is great! Next it's time to put a PWM throttle onto the Arduino and control it from the Commodore... I'll tinker with graphical programming in C also.

15Apr/124

First CAN Node: LEDs and Sensors

Despite my layout being tiny, I've decided that my layout will need two nodes. One will do the sensing of trains + management of LEDs and the other will control the throttle, points and isolation. I've now finished the building and wiring of the first node. One note for below: I've used Frame IDs to differentiate the messages across the network; unique numbers are used to define the message type and each node will have filters applied to only receive nodes that are specifically for them.

Detecting Trains

Train sensing has been implemented via light sensors. I needed 24 sensors around the track and decided that multiplexing was the only way to read them all. 3x 4051 ICs came in handy and were configured as per the schematic below. You can find a better explanation on light sensor implementation in the article here.

LED+Sensors CAN Node

I've simplified the code for reading the sensors as I'll ensure all of them have sufficient lighting and therefore 'stable' values. The circuit waits a second after power-up to read the initial sensor values. It then marks each one as 'active' if the value is greater than 20 'points' above the initial value. If a reading comes in lower then the initially read value then that value is lowered accordingly. I've also created a special command (frame id: 0x333) to allow a reset of all sensor values (specifically if ambient light proves a problem.)

The node sends out the sensor values and then pauses 100ms. The CAN packet with sensor data has the frame ID 0x111 and then fills three bytes with the 24 sensors states. A '1' is used for occupied and a '0' for vacant. The message will be received by any node on the network with the appropriate filters set.

Controlling LEDs

To control the LEDs I've re-used the previous MAX7219 ICs that I had lying around. There's a much more in-depth article here on how to use this chip. The MAX7219 requires three wires to the Arduino... but I'd already used up all the digital pins. Fortunately you can use the Analog pins just as easily.

LEDs on new layout LEDs on new layout

As the MAX7219 can receive bytes per row to control the LEDs I simply pass through the data from the CAN bus when appropriate. It would only take one CAN message as they contain 8 bytes of data. The Frame ID for this message is 0x222. You can see that I'm being pretty wasteful with Frame IDs but, fortunately, I don't have many specific messages to send.

Completed Sensor+LED Node

So, it was soldered and wired... quite a mess really. The CAN Controller was on-board but there was no 'direct communications' interface to the PC. This meant I had to pop the chip out (atmega328) each time I wanted to change the code... tedious and dangerous!! I managed to bend pins and confuse myself every now and then. Don't do this at home!

SENSOR+LED Node

Controller Node

I've got a third node hooked up to the computer via my trusty old Arduino Mega. This node transmits data from the CAN Bus to the PC via the standard serial-over-USB connection. A .NET application then shows the layout and allows you to interact with the LEDs and sensors accordingly. It shows the status of the sensors at any point in time and allows you to create rules in a sequence which can determine the speed and direction of the train.

The physical design of this node is simply the Arduino Mega plus the CAN schematic as seen in my previous blog post on implementing the MCP2515 controller.

  if (CAN.buffer0DataWaiting()) {
    CAN.readDATA_ff_0(&length,frame_data,&frame_id, &ext, &filter);
    Serial.write("X");
    Serial.write(frame_data[0]);
    Serial.write(frame_data[1]);
    Serial.write(frame_data[2]);
  }
  
  while (Serial.available() >= 9) {
    // read the incoming byte:
    b = Serial.read();
    if (b == 'L') {
      for (int i = 0; i < 8; i++) frame_data[i] = Serial.read();
      frame_id = 0x222;
      CAN.load_ff_0(8, &frame_id, frame_data, false);
    } else if (b == 'D') {
      for (int i = 0; i < 8; i++) frame_data[i] = Serial.read();
      frame_id = 0x666;
      CAN.load_ff_0(8, &frame_id, frame_data, false);
      //printBuf(frame_id, frame_data);
    } else if (b == 'R') {
      for (int i = 0; i < 8; i++) frame_data[i] = Serial.read();
      frame_id = 0x333;
      CAN.load_ff_0(8, &frame_id, frame_data, false);      
    } else if (b == 'P') {
      for (int i = 0; i < 8; i++) frame_data[i] = Serial.read();
      frame_id = 0x777;
      CAN.load_ff_0(8, &frame_id, frame_data, false);            
    }
  }

LED+Sensor Node Code

The sensor code consisted of an initial sensor read and then constant sensor checking. The sensors were then determined to be either 'occupied' or 'vacant' and then the data was sent. There are no 'smarts' here... the sensor data is simply hammered over the network.

The receive buffers are also checked for a Frame ID of 0x222 or 0x333. The filters are set appropriately to ensure that only these messages trigger the interrupts. The LED data is sent to the MAX7219 controller on the 0x222 message and the sensors are reset on 0x333.

void loop() {	
  send_data[0] = 0;
  send_data[1] = 0;
  send_data[2] = 0;
  for (sens = 0; sens < 8; sens++) {
    setOutputBit(sens);
    sensor_read = readInput(0);
    if (sensor_read < sensor[sens]) sensor[sens] = sensor_read;
    send_data[0] |= (((20 + sensor[sens])      <= sensor_read) << sens);
    sensor_read = readInput(1);
    if (sensor_read < sensor[sens]) sensor[sens] = sensor_read;
    send_data[1] |= (((20 + sensor[sens + 8])  <= sensor_read) << sens);
    sensor_read = readInput(2);
    if (sensor_read < sensor[sens]) sensor[sens] = sensor_read;
    send_data[2] |= (((20 + sensor[sens + 16]) <= sensor_read) << sens);
  }
  frame_id = 0x111;
  CAN.load_ff_0(3, &frame_id, send_data, false);

  frame_data[0] = 0;

  data1 = CAN.buffer0DataWaiting();
  data2 = CAN.buffer1DataWaiting();

  if (data1 || data2) {
    if (data1) CAN.readDATA_ff_0(&length, frame_data, &frame_id, &ext, &filter);
    if (data2) CAN.readDATA_ff_1(&length, frame_data, &frame_id, &ext, &filter);

    if (frame_id == 0x222) {
      for (int i = 0; i < 8; i++) lcl.setRow(0, i, frame_data[i]);
    } else if (frame_id == 0x333) {
      for (sens = 0; sens < 8; sens++) {
        setOutputBit(sens);
        sensor[sens] = readInput(0);
        sensor[sens + 8] = readInput(1);
        sensor[sens + 16] = readInput(2);
      }   
    }
 }
 delay(100);
}

Controller Node Code

The controller was the middle-man for translating PC code to the CAN bus and vice-versa. Sensor data from the CAN bus was converted to serial data with a leading 'X' byte. On the contrary LED commands were received from the PC with a leading 'L' byte. The following 8 bytes were sent as a single message onto the CAN bus once this character was seen.

    using System.IO.Ports;
    SerialPort _serialPort;

    private void SetupComPort() {
        _serialPort = new SerialPort("COM3", 9600, Parity.None, 8, StopBits.One);
        _serialPort.Handshake = Handshake.None;
        _serialPort.DataReceived += new SerialDataReceivedEventHandler(HandleData);
        _serialPort.Open();
    }

    private void cb_CheckedChanged(object sender, EventArgs e)
    {
        Byte[] bytesToSend = new Byte[9];
        bytesToSend[0] = (byte)'L';
        for (int i = 0; i < 8; i++) {
            for (int z = 0; z < 8; z++) bytesToSend[i+1] |= (byte)((LEDs[(i*8) + z].Checked ? 1 : 0) << z);
        }
        _serialPort.Write(bytesToSend, 0, 9);
    }

    private void UpdateSpeed()
    {
        Byte[] bytesToSend = new Byte[9];
        bytesToSend[0] = (byte)'D';
        bytesToSend[1] = (byte)(direction ? 1 : 0);
        bytesToSend[2] = (byte)hSpeed.Value;
        bytesToSend[3] = (byte)points[0]; //for loop? probably should.
        bytesToSend[4] = (byte)points[1];
        bytesToSend[5] = (byte)points[2];
        bytesToSend[6] = (byte)points[3];
        bytesToSend[7] = (byte)points[4];
        bytesToSend[8] = (byte)points[5];
        _serialPort.Write(bytesToSend, 0, 9);
        Console.WriteLine("Sent: D" + bytesToSend[1] + "-" + bytesToSend[2]);
    }

Notes learnt from hooking all this up

  • CAN Filters work, but you can still see messages
    On the CAN controller you can set filters to only allow specific messages through. It actually turns out that, regardless of the filter being set, if you constantly poll either buffer for a message that, if there has been a message sent, it will be available and can be received from the network. Filters simply prevent the message from triggering the 'message waiting in buffer' interrupts. So, the best bet is that you check for a message in a buffer first rather than simply constantly attempt to receive messages. Let the CAN IC tell you that a message is waiting!
  • You can easily spam a CAN network
    Sending too many messages over the network will delay or prevent transmission of other nodes' messages. One node can send too many messages over the network preventing another from successfully getting messages into a third nodes' buffers. It's all down to how quickly the third node can receive the messages. If both receiving buffers are full then the message will fly on past and not be seen.
  • Inserting and removing ICs
    PLEASE only do this when you need to and do this with care! Lever them out from both ends and DO NOT just use your fingers. Insert them evenly as well. It's too damn easy to destroy these delicate and expensive components.

...next will be a node to control the thottle, isolated blocks and point-servos.

3Mar/1273

Arduino + Controller Area Network (CAN)

Stolen from Wikipedia: The CAN (controller area network) bus is a vehicle bus standard designed to allow microcontrollers and devices to communicate with each other within a vehicle without a host computer. This bus standard includes it's own messaging protocol for communications between nodes on the network.

One of the most popular uses for this technology is in the automobile where the network provides a communication channel between the ECU, Transmission, airbags, braking systems, etc...

As you'll have noticed, I've recently been investigating OpenLCB which uses this technology for communications between its nodes on a model railway. The article below will show how the CAN bus can be incorporated with an Arduino to allow communications between different microcontrollers/nodes on your network.

Topography

The OpenLCB project dictates that the CAN bus implemented via their standards must not be in the form of a ring. When I initially saw the Railstars:io prototype I expected that a ring would be required; the prototype shows an 'in' and 'out' communications port. Little did I realise that, upon closer inspection, the io could act as a termination point for the CAN bus that it connects to. We therefore will implement a layout as follows on our railways...

can-setup

The OpenLCB project will use standard RJ45 cables (i.e. ethernet cables) due to the requirement of twisted pair technology to guarantee data transmission.

Interfacing with the Arduino via SPI

The Serial Peripheral Interface is a communication standard used to provide communications between integrated circuits. The Arduino has an SPI library and therefore allows us to easily communicate to the CAN interface.

We'll be using the MCP2515 and MCP2551 integrated circuits to transmit/receive our data from SPI onto the CAN bus. This setup has been done before in Sparkfun's CAN-BUS Shield, the CAN-CAN, the Universal CAN Library (Universelle CAN Bibliothek) and the CANduino. The CAN-CAN Schematic and Sparkfun's schematics provide a great reference point for hooking the whole lot up. Of course, you could also just grab their shield if you don't want to build everything yourself.

Schematic

The image below should be pretty self-explanatory. See the next section for the exact pins for the SPI interface. RESET and INT can go to any digital pin.

MCP2515

Building, troubleshooting and talking to one's self

I built two at the same time to be able to test node-to-node communications. There was nothing overly tricky in the construction... I just chose to use relatively tight pcb prototyping boards which didn't leave much room for error. The above schematic was followed and the devices were hooked up to my Arduino(s).

Initial board

I tried the loopback test from the canduino project and got random responses... the data going in was not the same as the data coming back. I started the debugging process after letting out a little sigh.

It turns out that, on all Arduinos, the SPI interface can only be used on specific pins:

SPI Pin Arduino Mega Smaller Arduinos (168,328)
SS/CS 53 10
MOSI 51 11
MISO 50 12
SCK 52 13

Note that the CAN library also then required a change as the pins in it are hard-coded to a 168/328 Arduino and wouldn't have ever worked with the Mega. You can find my fix for this down below.

I finally had responses after the pins were in the correct locations... but not the responses I wanted to see; I looked at the crystals next. My Arduinos all used 16.000MHZ crystals, but my local hobby shop only had 4mhz, 8mhz or 20mhz. This concerned me and I google'd and google'd to work out if, in the same circuit, multiple ICs could be driven off different oscillation rates. I decided that, since the 'SPI' interface had it's own 'bus rate' and the CAN bus also had it's own 'bus rate' that the crystals therefore did not affect these speeds. Also, the Arduino would dictate the speed of the SPI interface since it was the master (and slower.) I therefore bought 20mhz crystals for the MCP2515 chips, with my fingers crossed. Of course, this could have been the next issue.

Fortunately, from my previous attempts at barebones Arduinos, I had a 16mhz crystal on-hand. I swapped out the 2 capacitors and the 20mhz for the 16mhz and hooked it all up again. Nothing...

So, I then whipped out my Atmega328p 'barebones' Arduino and hooked it up to that. The SPI pins were different, as per above, so I made sure they were correct... and the CAN library from the canduino source had to be modified back to the required pins. After putting it all together I checked the serial output in the Arduino Serial Monitor and saw the correct response. The loop back test was working!

WTH... there was no reason that it should... except that maybe the SPI interface on my Mega was fried? I then, just because I hate caution and feed it to the wind all the time, swapped the 20mhz crystal and capacitors back in. It STILL WORKED! Good... I could keep the 16mhz for my other barebones Arduino.

So... with a known-working CAN node, I swapped it back onto the Mega. WTH... it worked. I now had my CAN nodes (by this time the other node built was also functioning thanks to it's guinea pig brother) talking to themselves. It was time to get them to talk to each other!

Talking in the CAN

Hah... there's an unwritten rule to not talk whilst in the toilet, but in our situation we'll make an exception. From the start I'd built the bus and was just itching to get the damn thing transmitting messages. The setup was simple: terminal blocks were used at junctions and standard dodgy speaker wire was the main network cable. 120ohm resistors were the terminators as per the standard CAN specification.

Terminators One node setup

The only real issues here were the source code (or my lack of knowledge of how it worked) and the usage of crystals. Yes, they came back to haunt me... it turns out that, whilst I had the loopback going, I'd left a 16mhz on one node and a 20mhz on the other. In the source code you specify the 'CAN Bus Speed' which, of course, was configured the same on both Arduinos. The actual issue was that, when the Arduino told the CAN controller to run at bus speed '1000', the CAN nodes did this but, due to their differing crystals, their calculations of what '1000' meant were incorrect! Since one was running 4mhz faster (20mhz vs. 16mhz) it must've been communicating on the bus at a different rate and therefore confusing the hell out of the other node.

The next issue was the code... I looked over it and thought that each node would take turns in transmitting and recieving... I was wrong: you had to actually set one as the sender and one as the receiver. I configured this based on the Arduino class (by #IFDEF just as the SPI pins were configured) and then had communication! My nodes were live and functional!

Communicating More communications

RX/TX LEDs

The final step was wanting RX/TX activity LEDs. It turns out that the MCP2515 supports this, but you need to do a little configuration. If you view the datasheet at microchip you'll note that the RXnBF Pins can be configured as 'nothing', 'general outputs' or 'low on RX buffer full'. I was hoping that obe would be 'high on TX buffer', but no such luck. It actually turns out that the chip has two RX buffers and therefore two RXnBF Pins. LEDs attached to these pins will, once configured as per below, illuminate once the associated RX buffer is full. If you ever see both full then you might be losing data?

RX TX LEDs

Second option: It turns out that in the MCP2515 Demo Board PDF that you can just put two LEDs on the RXCAN(1)/TXCAN(2) pins of the MCP2515.

Third option: It seems that you can also configure the INT pin to fire on TX buffer 'emptying'. You'll have to disable the pin firing on all other error situations to do this. By default, this pin is usually pulled 'high' by the MCU, so you must either disconnect that, use a resistor between the MCU and MCP2515 or just pull the pin high normally via a pullup resistor and then connect an LED in parallel with this to the INT pin. I imagine that in more complicated scenarios you'd actually want to know once the MCP2515 has generated an interrupt.

IDs: Standard or Extended?

All messages sent on the network have a 'frame id' of either (standard mode) 11-bits or (extended mode) 29-bits. It must be understood that this 'frame id' does not actually identify a node! It is simply an identifier for the actual message being transmitted. This identifier can contain any form of information you want; the bits are yours to play with.

One main use for this identifier is to record the intended recipient of the message you are sending. Depending on the amount of nodes in your network (don't forget to think of future scalability!) you can use anywhere up to 29-bits to uniquely identify them. Of course, this identifier could just be used to identify the 'class' of node and then you could put further data in the message to indicate the exact recipient. You could also go the other way and only use a portion of the identifier for the recipient code and have the rest for other uses.

The MCP2515 has in-built filtering to work on the identifier of incoming messages. If you're only using standard mode then the filters also apply to the first two bytes of the message. The filtering works inside the MCP2515 and therefore doesn't saturate your SPI link as the messages are stopped prior to being put into the RX buffer.

MCP2515 Modes

There's 5 modes that you can set the MCP2515 into. These are as follows:

Mode Description
Configuration This mode allows the developer to update settings. The following settings require this mode to be set first: CNF1,CNF2,CNF3,TXRTSCTRL,MASKS,FILTERS.
Normal Standard operating mode. RXM0 and RXM1 per buffer do not apply in this mode. All filters and Masks do apply.
Sleep Low-power mode. SPI remains active, and the MCP2515 will wake up if there is activity on the CAN bus if the WAKIE/WAKIF interrupts are enabled. After woken, the chip will be in Listen Mode.
Listen Make sure there are two or more nodes on the CAN bus. Whilst in Listen mode, the chip won't output a thing, so if another chip sends a message then the message will constantly circulate on the bus if there is no other node to 'ACK' it. Another node must be there to stop the message transmission. RXM0 and RXM1 per buffer apply in this mode and the masks/filters also do too.
Loopback All messages sent are returned via the RX buffers, filtered first. No messages are sent out over the CAN. No messages are received from the CAN.

Buffers

It needs to be noted that the MCP2515 has two receiving buffers known as Buffer 0 and Buffer 1. Messages will go to the first if it's free (and not filtered) and then the second. Filters can be set on both, but the second buffer is much more flexible with four filters.

I don't know the exact reason for the MCP2515 containing two buffers, and then why one is more flexibile than the other, but in the end it's up to the developer as to how to use them. Filters can be configured to allow specific message 'types' on each buffer. This then means that, for example, the developer could configure the second buffer to only allow emergency messages such as 'STOP THAT TRAIN' whilst the first deals with the standard communications.

Filtering CAN Messages via the MCP2515

With the base configuration, an MCP2515 controller will accept any messages sent across the network. This therefore means that your nodes need to filter each individual message coming across the bus to determine if they themselves are the intended recipient. Fortunately the MCP2515 has in-built configurable filtering to allow only messages intended for the node to be passed through its RX buffers.

The MCP2515 has 6 configurable filters with 2 masks. The first mask works in conjunction with the first two filters and applies to the first receiving buffer whereas the second mask works with the final 4 filters and works with the second receiving buffer. Filtering is enabled when any bit of a mask is '1' and disabled when the masks are completely zero'd out.

You may be wondering why both a mask and a filter are needed? As mentioned above, you may not have used the entire 'identifier' area of the sent message to indicate the recipient. Therefore, when you're checking an incoming message, you wont want to filter the entire identifier. As the identifier bits can be a '1' or '0', you need to initially specify which bits you want to check (mask) and then whether or not they are a '1' or '0' (filter).

Node 1 ('101') Node 2 ('110')
Identifier 10110 10110
Mask 00111 00110
AND'd 00110 00110
Filter 00101 00110
Match? False True

In the table above, you can see that Node 1 failed to match. The filter applied was looking for '101' (5 in decimal) but the lower 3 bits identifier (that's what the mask was looking for) was actually '110' (6 in decimal). You can see that the actual result was '110' after the mask was applied to the identifier. The second node matched as the filter is actually looking for '110'.

Now, we have to use the full registers when we apply this to the MCP2515. This means filling out 29bits of data. When in doubt, fill everything else with zeroes. Each register in the MCP2515 is 8bytes, this therefore means that you need 4 bytes per filter and 4 bytes per mask. The 6 filters and 2 masks that were mentioned above are located in the following areas:

Mask 0 Filter 0 Filter 1 Mask 1 Filter 2 Filter 3 Filter 4 Filter 5
0x20 0x00 0x04 0x24 0x08 0x10 0x14 0x18
0x21 0x01 0x05 0x25 0x09 0x11 0x15 0x19
0x22 0x02 0x06 0x26 0x0a 0x12 0x16 0x1a
0x23 0x03 0x07 0x27 0x0b 0x13 0x17 0x1b

Right, now that you know you need 4 bytes for a filter and then 4 bytes for a mask... and you know where to store them... you'll probably now need to know how to construct a mask and filter. We'll start with a mask that looks for the value 11 (decimal!) [0x0b in hex, 1011 in binary] in the standard 11-bit identifier. As you can guess, I've just provided the filter by representing the value in binary; we simply need to zero-out the bits to the left to ensure we provide the correct number. Have I provided the mask? No! Look at the truth table below if we were to use 1011 as the mask.

Node 1 ('11011') Node 2 ('01111') Node 3 ('01011')
Identifier 11111 11011 01011
Mask 01011 01011 01011
AND'd 01011 01011 01011
Filter 01011 01011 01011
Match? True True True

They all matched!?!?!? Since we were only checking the 'exact' value via the mask, we weren't actually looking to see if any of the bits around the value were set. The basic principal is that we need to know the maximum length of the value we could be looking for and then use that as the mask. For example, if you have up to '11111' (32 including '0') ids on your network, then you want to make sure that your mask is '11111' and that your filter is the exact number of the node checking if the message is theirs.

Node 1 ('01011') Node 2 ('01111') Node 3 ('11011')
Identifier 11111 11011 01011
Mask 11111 11111 11111
AND'd 11111 11011 01011
Filter 01011 01011 01011
Match? False False True

That's better! Make sure your values are correct! So, in the example below we'll use '11111' as the max value and therefore the mask. We'll then use 01011 as the value of the node we're pretending to be and we'll insert these into the correct registers (being Mask 0 and Filter 0.)

Message Acceptance Process

So, two buffers with differing filters... what happens when a message arrives? The basic idea is that a message will hit the first buffer and, if not accepted, attempt to hit the second buffer. The message is then discarded if the second buffer is also configured to not accept this message.

Note: There are also two bits (RXM0 and RXM1) in the RXBnCTRL (where n is either 0 or 1 depending on the buffer) that determine how messages are accepted. Note: My code currently has the ability to set these registers; for the life of me I cannot actually get the chip to function as per the datasheet when these registers are set. If anyone else has succeeded in using the RXM bits successfully then please leave a comment and tell how you did it!

RXM0 RXM0 Description
0 0 [Default] Enable reception of all valid messages as determined by the appropriate acceptance filters. (See the pseudo-code below as to how the masks and filters apply.)
0 1 Only accept standard messages. Filters are useless if the EXIDE bit does not match with this value.
For example, if you put an extended filter on buffer 0 but you disable extended messages to buffer zero with this configuration then that filter will never be used.
1 0 Only accept extended messages. The same rules apply that any standard filters on a buffer with this setting will not be used.
1 1 Receive ALL messages regardless of filters. Also recieve messages that have caused an error during transmission. The fragment of message received in the buffer prior to the error will be brought through. This mode should really only be used for debugging a CAN system.

So, as long as the RXM bits are both set to '0', the message buffers will then process through the following procedure to accept or deny messages.

  for EACH BUFFER b: (initially 0)
    if (MASK 'b' is ZERO) { //mask 0 for buffer 0 and mask 1 for buffer 1
      Accept new message on Buffer b
    } else {
      using FILTERs 0,1 (if b equals 0) OR FILTERs 2,3,4,5 (if b equals 1)
        if (both the message and filter are 'extended') or (both the message and filter are 'standard') then
          for each '1' in MASK as 'x', does FILTER[x] equal message[x]?
            if all match then accept message
            else check next filter as this filters bits do not match the relevant message bits.
        else check next filter as this filter type does not match this message type.
      if still not accepted then TRY BUFFER 1
    }

A mask is disabled when all its bits are '0'. As soon as a bit is '1' then that mask applies to the relevant buffer. As per the section above, the bits in the filter are then checked against the relevant message bits based on what bits the mask specifies the process to check.

On the contrary, a filter is not 'disabled' when all bits are '0'; instead you have configured two filters, on the buffer related to the mask, which will only permit standard messages where the identifier is zero. If you set the mask on a buffer to anything other than 0 and then don't touch the filters, this will prevent extended messages from coming through (as both filters have '0' as the EXIDE bit) and also prevent any standard messages that do not have an identifier of zero.

In the event that you don't want to receive a specific message type, you must put a mask on both receiving buffers and then an appropriate filter to block out the specific identifier bits. On the contrary, make sure that your masks and filters are smart enough to capture all other data required for your CAN node.

If in doubt, write an 'all-seeing' debugging application to view all traffic on your bus... it'll allow you to diagnose why messages are landing where you might not want them to. As per above, setting both RXM bits to 1 will bypass your filters; this way you don't have to muck around with your own configuration too much. Just make sure you have more than two other nodes on the bus if you want another in Listen mode. Having one node on the bus, or two with one in listen mode, will mean that any message transmitted will stay perpetually on the wire. A message must be consumed by another node for it to disappear.

Storing Masks and Filters on the MCP2515

Scroll up to see the register addresses for the masks and filters. The contents of each of these registers looks as follows:

Filter/Mask Bit 7 Bit 6 Bit 5 Bit 4 Bit 3 Bit 2 Bit 1 Bit 0
Byte 0 (RX[F|M]nSIDH)* SID10 SID09 SID08 SID07 SID06 SID05 SID04 SID03
Byte 1 (RX[F|M]nSIDL)* SID02 SID01 SID00 –– EXIDE/––** –– EID17^ EID16^
Byte 2 (RX[F|M]nEID8)* EID15 EID14 EID13 EID12 EID11 EID10 EID09 EID08
Byte 3 (RX[F|M]nEID0)* EID07 EID06 EID05 EID04 EID03 EID02 EID01 EID00
  • *: 'n' is the filter number, i.e. RXF0SIDH is the 'Standard ID High Receive Register Filter 0'.
    [F|M] means that the name is 'RXF1SIDH' for Filter 1 and 'RXM1SIDH' for Mask 1.
  • **: 'EXIDE': set this to '1' and this filter will only apply to Extended IDs, '0' means it will only apply to standard IDs.
    This byte is unimplemented for Masks. Just leave it as zero.
  • ^: These two bits are not used if you are in standard mode as you only need 16 bits to check the first two bytes of data.

Have I lost you yet? Each filter and mask needs to be able to store both the Standard ID bits and the Extended ID bits. As mentioned before, if you're in standard mode then the 'lower' two bytes (i.e. bits 16 to 0 [17,18 are ignored!]) of the extended area are for matching against the first two bytes of data. You can disable this by ensuring that the associated mask has zeroes in the extended area. On the contrary, if you wanted to check the first byte in the data for a specific value then you would pass in '001111111100000000' as the extended area of the mask with the relevant extended bits set in filter.

Somewhere up above I mentioned we'd use '11111' for the mask and '01011' for the node id. We'll now store this in the registers. We'll use Mask 0 and Filter 0 for this and we'll make sure the rest are all zeroes so they don't apply. You'll note that the first register (RXM0SIDH) starts with zeros; as per above this means that the identifier could actually contain any data in this area and, since the mask is '0' for these bits, the acceptance process wont care at all! The registers will need to be set as follows:

Register Data Register Data
RXM0SIDH [0x20] 0b00000011 RXF0SIDH [0x00] 0b00000001
RXM0SIDL [0x21] 0b11100000 RXF0SIDL [0x01] 0b01100000
RXM0EID8 [0x22] 0b00000000 RXF0EID8 [0x02] 0b00000000
RXM0EID0 [0x23] 0b00000000 RXF0EID0 [0x03] 0b00000000
Note: All other Registers in the RXF/RXM space need to be zero'd out!

So, as you can see above, we've put the mask 0b00000011111 in the 'standard id area' of Mask 0 and 0b00000001011 in the 'standard id area' of filter 0. Based on our logic above this will then only allow messages through that match the above requirements. Below is the code to actually do this on the Arduino.

#define MASK_0 0x20
#define FILTER_0 0x00
void CANClass::setMaskOrFilter(byte mask, byte b0, byte b1, byte b2, byte b3) {
	setMode(CONFIGURATION);  
	setRegister(mask, b0);
	setRegister(mask+1, b1);
	setRegister(mask+2, b2);
	setRegister(mask+3, b3);
	setMode(NORMAL);
}

//set MASK 0 for RXB0 a mask checks all bits of the standard id.
CAN.setMaskOrFilter(MASK_0,   0b11111111, 0b11100000, 0b00000000, 0b00000000);
//set Filter 0 to 0x555. Therefore only messages with frame id: 0x555 are allowed through this buffer. 
CAN.setMaskOrFilter(FILTER_0, 0b10101010, 0b10100000, 0b00000000, 0b00000000);

Note that only setting this for buffer 0 will mean that any message that doesn't match will come through on buffer 1. You'll need to also set a mask on buffer 1 to stop the messages completely. Of course, you can also only ever check buffer 0 for messages and not care if buffer 1 has anything waiting.

Filters in action: Guessing game.

Ok, who would've thought you could make microcontrollers play a game together. You know the old trick, someone else chooses a 'random' number and you get to 'randomly' guess it. Why don't we make our Arduino's play the same game?

We'll set the filter on buffer 0 to the number the receiver is thinking of. This means that the sender will only actually get a message through to the receiver if they guess the correct identifier. Both controllers will then swap roles. Just to make life easier we'll limit the range of numbers allowed, but we'll still make the guessing as 'random' as possible.

I could paste the full code here, but instead I'll just post the general idea.

Main setup
Set Mask of Buffer 0 to the 11-bit standard identifier
Set Mask of buffer 1 to the same 11-bit id
set filter 2 (first filter of buffer 1) to a specific ID for messages that aren't guesses.
Set one node to RX and one to TX
Initialise the random generator.
Receiver Role Transmitter Role
  1. Randomly set 'guess' number from known range.
  2. Set Filter 0 to guess number.
  3. Sit and wait for transmitter to guess...
    - if they send a message on buffer 1 then pass it on to the host.
    - if they get through on buffer 0 then it's their turn.
  1. Randomly choose a number to guess.
  2. Set the frame id of the message and send.
  3. Wait for response, if found then we won, swap to receiver.
  4. If no response, send guessed number to buffer 1 of receiver.
  5. Loop and guess another number if required.

Source code (including my version of the CAN Library)

The final piece of the puzzle is always the source code. I've included the guessing game example as well. Note that my code is based off the canduino and I intend to send changes upstream where possible to make life easier for everyone.

Download the library and example here.
29Feb/129

Arduino + Thermal Printers (Sparkfun, IBM 4160-TF6)

I've always wanted to print my own receipts... devious activities come to mind; but the following usage of receipt printers is nothing sinister at all. My goal is to print out activities for trains on the layout; certain locations will have a new reporting mechanism!

I'd seen a few printers on eBay, most being USB. Serial was always to be my preferred method and I had the components on hand. I then saw the Thermal Printer at Sparkfun and decided that it'd be my first guinea pig.

Sparkfun Thermal Printer

The Sparkfun themal printer is, like most of their products, targetted at the Arduino. Thanks to this there is a wealth of information on their page on how to connect and use it. After hooking up a proper power supply (they recommend 9v @ 2Amps, so use a 7809 with a heatsink!) the printer just started spewing out whatever I threw at it.

Sparkfun Thermal Printer

The easiest way to use this is via the library found here: Displaying on Paper – Thermal Printer + Arduino. bildr.org is actually a really cool site full of interesting projects for the Arduino, check it out when you can!

There's a forum for discussing the above library, in which I've already posted my praises to the developer. If you need a hand then go over and ask away... they're always open to feedback and improvements to the library too!

Note that this printer uses small rolls of 58mm paper. I found a 10-pack of these at OfficeWorks (Australia) for AUD$9.95.

IBM 4610-TF6 (on Windows)

I'd finally found a dirt cheap printer on eBay that was RS-232. It was a chunky/retro IBM thermal printer and really was just a larger, more robust version of the Sparkfun thermal printer above. It didn't come with a power supply and, after purchasing, I realised that it wanted 38v! What the hell? It seems that the 'thermal' side of it uses a lot of current to burn the paper! It also wanted 3 AMPS at 38v... where the hell would I find that?

Split open IBM SureMark 4610-TF6 Bypassing the power socket 58mm paper output!

I installed the printer drivers here on a Windows XP 32-Bit machine (DOES NOT WORK WITH 64-Bit!) and provided it 12v @ 2A. The LED came on, but the printer showed up as 'offline'. All attempted connections via Hyper-Terminal showed the port as 'already open'; the printer driver would've been the cause. I uninstalled it and rebooted, but Hyper-Terminal wasn't receiving any responses after connecting successfully.

Middle bar not printing! Bit Switches

I then found a second 12v power supply and chained them together... Prior to this I'd re-installed the printer drivers and, upon the boost to 24v, the printer flicked to 'online' and the LED glowed slightly brighter. I could now also use the line-feed button on the top of the printer! I opened notepad, loaded a text file from the Arduino directory and, without thinking, hit print. The printer control panel window showed the job spooling up to 20 pages and then the printer started .... and kept going ... for an hour. It printed at a rate of about 1 line per 5 seconds.

Testing output

I had no idea how to cancel it... so I had to let it go.

IBM 4610-TF6 (on the Arduino)

It was now time to make this printer talk to my Arduino Mega. I hooked it all up based on how the Sparkfun printer wiring and attempted to use the same code; nothing happened. I then used the code from Tom Taylor's Microprinter blog post. Once uploaded, I had junk coming out of the printer... It occurred to me that I probably needed a MAX232 in the middle to raise the voltages to proper RS232 levels (as per everyone elses examples!)

I hooked up the MAX232 as per the schematic below and then had a functional printer from my Arduino! Determining the actual commands to send it came next.

Printer Connection to Arduino Proper cabling Initial DB9 Connections
MAX232 Setup Two RS232 cables for the Arduino

Fonts, spacing, etc...

Right, this gets tricky... you can either put 58mm or 80mm paper in this printer. 80mm is recommended as it has better chances of staying aligned with the paper cutter (coming out diagonally is actually an issue.) I had started with the 58mm but quickly went and bought 80mm paper (AUD$12 for 4 80x80 rolls) as I also wanted the extra printing space.

You can work out how many characters per line based on the font chosen, character spacing. It is pretty much expected that you're using 80mm paper. Font A is 10x20, Font B is 12x24 and Font C is 8x16. The Cash Receipt print line is 72 mm (2.83 inches) long. There are 576 dots per line and 203 dots per inch.

The Application Guide provides the following calculations:

  • 20 CPI ⇒ 8-dot wide character + 2-dot space (Font C) ⇒ 57 characters/line
  • 17 CPI ⇒ 10-dot wide character + 2-dot space (Font A) ⇒ 48 characters/line
  • 15 CPI ⇒ 10-dot wide character + 3-dot space (Font A) ⇒ 44 characters/line
  • 12 CPI ⇒ 12-dot wide character + 5-dot space (Font B) ⇒ 33 characters/line

More test output

Printing a Barcode

In a comment below, Jonas has asked how to print a barcode from any programming language via the serial port... Here's a list of instructions to do so:

  1. Set the Font (0x00 or 0x01):
    0x1d 0x66 FONT 0x0a
  2. Set the Text Location:
    • 0x00 - None
    • 0x01 - Above
    • 0x02 - Below
    • 0x03 - Both
    0x1d 0x66 POSITION 0x0a
  3. Set Barcode Width:
    0x1d 0x68 WIDTH 0x0a
  4. Set Barcode Height:
    0x1d 0x77 HEIGHT 0x0a
  5. Print a Barcode. 'CHARS' is the list of characters to print.
    • 0x00 - UPC-A
    • 0x01 - UPC-E
    • 0x02 - JAN13 (EAN)
    • 0x03 - JAN8 (EAN)
    • 0x04 - CODE 39
    • 0x05 - ITF
    • 0x06 - CODABAR
    • 0x07 - CODE 128 (c)
    • 0x08 - CODE 93
    • 0x09 - CODE 128 (a, b)
    0x1d 0x6b BARCODE_TYPE CHARS 0x00

A 'Hello World' example of point 4 would be:

0x1d 0x6b 0x00 0x48 0x65 0x6C 0x6C 0x6F 0x20 0x57 0x6F 0x72 0x6C 0x64 0x00

IBM SureMark Thermal Printer Library for Arduino 1.0

IBM provides a great reference document for this collection of printers: Application interface specification for IBM 4610 printers. I found it to be a little hit-and-miss as to what commands are available on the TF6, but most worked well. Either way, I built the following library which provides the following functionality:

  • Text Styles: Bold, Underline, Overline, Inverted, Arbitrary Font Scaling, Double Height/Width, Rotated, Upside-down
  • Barcode Printing: UPC-A, UPC-E, JAN13 (EAN), JAN8 (EAN), CODE 39, ITF, CODABAR, CODE 128 (c), CODE 93, CODE 128 (a, b)
  • Image Printing from data stream, image storing to printer RAM and image printing from printer RAM
  • Message storing to printer RAM and printing from printer RAM
  • Beeper sounds. (Example below has 'Mary had a little lamb')
  • Paper cutting, line spacing, line feeding, etc...
Download the library and example here.

Other references

It turns out that, if I'd google'd more, I would've found a lot more help around the traps... here's a few locations to check out: