Subscribe via RSS

First CAN Node: LEDs and Sensors

Despite my layout being tiny, I've decided that my layout will need two nodes. One will do the sensing of trains + management of LEDs and the other will control the throttle, points and isolation. I've now finished the building and wiring of the first node. One note for below: I've used Frame IDs to differentiate the messages across the network; unique numbers are used to define the message type and each node will have filters applied to only receive nodes that are specifically for them.

Detecting Trains

Train sensing has been implemented via light sensors. I needed 24 sensors around the track and decided that multiplexing was the only way to read them all. 3x 4051 ICs came in handy and were configured as per the schematic below. You can find a better explanation on light sensor implementation in the article here.

LED+Sensors CAN Node

I've simplified the code for reading the sensors as I'll ensure all of them have sufficient lighting and therefore 'stable' values. The circuit waits a second after power-up to read the initial sensor values. It then marks each one as 'active' if the value is greater than 20 'points' above the initial value. If a reading comes in lower then the initially read value then that value is lowered accordingly. I've also created a special command (frame id: 0x333) to allow a reset of all sensor values (specifically if ambient light proves a problem.)

The node sends out the sensor values and then pauses 100ms. The CAN packet with sensor data has the frame ID 0x111 and then fills three bytes with the 24 sensors states. A '1' is used for occupied and a '0' for vacant. The message will be received by any node on the network with the appropriate filters set.

Controlling LEDs

To control the LEDs I've re-used the previous MAX7219 ICs that I had lying around. There's a much more in-depth article here on how to use this chip. The MAX7219 requires three wires to the Arduino... but I'd already used up all the digital pins. Fortunately you can use the Analog pins just as easily.

LEDs on new layout LEDs on new layout

As the MAX7219 can receive bytes per row to control the LEDs I simply pass through the data from the CAN bus when appropriate. It would only take one CAN message as they contain 8 bytes of data. The Frame ID for this message is 0x222. You can see that I'm being pretty wasteful with Frame IDs but, fortunately, I don't have many specific messages to send.

Completed Sensor+LED Node

So, it was soldered and wired... quite a mess really. The CAN Controller was on-board but there was no 'direct communications' interface to the PC. This meant I had to pop the chip out (atmega328) each time I wanted to change the code... tedious and dangerous!! I managed to bend pins and confuse myself every now and then. Don't do this at home!

SENSOR+LED Node

Controller Node

I've got a third node hooked up to the computer via my trusty old Arduino Mega. This node transmits data from the CAN Bus to the PC via the standard serial-over-USB connection. A .NET application then shows the layout and allows you to interact with the LEDs and sensors accordingly. It shows the status of the sensors at any point in time and allows you to create rules in a sequence which can determine the speed and direction of the train.

The physical design of this node is simply the Arduino Mega plus the CAN schematic as seen in my previous blog post on implementing the MCP2515 controller.

  if (CAN.buffer0DataWaiting()) {
    CAN.readDATA_ff_0(&length,frame_data,&frame_id, &ext, &filter);
    Serial.write("X");
    Serial.write(frame_data[0]);
    Serial.write(frame_data[1]);
    Serial.write(frame_data[2]);
  }
  
  while (Serial.available() >= 9) {
    // read the incoming byte:
    b = Serial.read();
    if (b == 'L') {
      for (int i = 0; i < 8; i++) frame_data[i] = Serial.read();
      frame_id = 0x222;
      CAN.load_ff_0(8, &frame_id, frame_data, false);
    } else if (b == 'D') {
      for (int i = 0; i < 8; i++) frame_data[i] = Serial.read();
      frame_id = 0x666;
      CAN.load_ff_0(8, &frame_id, frame_data, false);
      //printBuf(frame_id, frame_data);
    } else if (b == 'R') {
      for (int i = 0; i < 8; i++) frame_data[i] = Serial.read();
      frame_id = 0x333;
      CAN.load_ff_0(8, &frame_id, frame_data, false);      
    } else if (b == 'P') {
      for (int i = 0; i < 8; i++) frame_data[i] = Serial.read();
      frame_id = 0x777;
      CAN.load_ff_0(8, &frame_id, frame_data, false);            
    }
  }

LED+Sensor Node Code

The sensor code consisted of an initial sensor read and then constant sensor checking. The sensors were then determined to be either 'occupied' or 'vacant' and then the data was sent. There are no 'smarts' here... the sensor data is simply hammered over the network.

The receive buffers are also checked for a Frame ID of 0x222 or 0x333. The filters are set appropriately to ensure that only these messages trigger the interrupts. The LED data is sent to the MAX7219 controller on the 0x222 message and the sensors are reset on 0x333.

void loop() {	
  send_data[0] = 0;
  send_data[1] = 0;
  send_data[2] = 0;
  for (sens = 0; sens < 8; sens++) {
    setOutputBit(sens);
    sensor_read = readInput(0);
    if (sensor_read < sensor[sens]) sensor[sens] = sensor_read;
    send_data[0] |= (((20 + sensor[sens])      <= sensor_read) << sens);
    sensor_read = readInput(1);
    if (sensor_read < sensor[sens]) sensor[sens] = sensor_read;
    send_data[1] |= (((20 + sensor[sens + 8])  <= sensor_read) << sens);
    sensor_read = readInput(2);
    if (sensor_read < sensor[sens]) sensor[sens] = sensor_read;
    send_data[2] |= (((20 + sensor[sens + 16]) <= sensor_read) << sens);
  }
  frame_id = 0x111;
  CAN.load_ff_0(3, &frame_id, send_data, false);

  frame_data[0] = 0;

  data1 = CAN.buffer0DataWaiting();
  data2 = CAN.buffer1DataWaiting();

  if (data1 || data2) {
    if (data1) CAN.readDATA_ff_0(&length, frame_data, &frame_id, &ext, &filter);
    if (data2) CAN.readDATA_ff_1(&length, frame_data, &frame_id, &ext, &filter);

    if (frame_id == 0x222) {
      for (int i = 0; i < 8; i++) lcl.setRow(0, i, frame_data[i]);
    } else if (frame_id == 0x333) {
      for (sens = 0; sens < 8; sens++) {
        setOutputBit(sens);
        sensor[sens] = readInput(0);
        sensor[sens + 8] = readInput(1);
        sensor[sens + 16] = readInput(2);
      }   
    }
 }
 delay(100);
}

Controller Node Code

The controller was the middle-man for translating PC code to the CAN bus and vice-versa. Sensor data from the CAN bus was converted to serial data with a leading 'X' byte. On the contrary LED commands were received from the PC with a leading 'L' byte. The following 8 bytes were sent as a single message onto the CAN bus once this character was seen.

    using System.IO.Ports;
    SerialPort _serialPort;

    private void SetupComPort() {
        _serialPort = new SerialPort("COM3", 9600, Parity.None, 8, StopBits.One);
        _serialPort.Handshake = Handshake.None;
        _serialPort.DataReceived += new SerialDataReceivedEventHandler(HandleData);
        _serialPort.Open();
    }

    private void cb_CheckedChanged(object sender, EventArgs e)
    {
        Byte[] bytesToSend = new Byte[9];
        bytesToSend[0] = (byte)'L';
        for (int i = 0; i < 8; i++) {
            for (int z = 0; z < 8; z++) bytesToSend[i+1] |= (byte)((LEDs[(i*8) + z].Checked ? 1 : 0) << z);
        }
        _serialPort.Write(bytesToSend, 0, 9);
    }

    private void UpdateSpeed()
    {
        Byte[] bytesToSend = new Byte[9];
        bytesToSend[0] = (byte)'D';
        bytesToSend[1] = (byte)(direction ? 1 : 0);
        bytesToSend[2] = (byte)hSpeed.Value;
        bytesToSend[3] = (byte)points[0]; //for loop? probably should.
        bytesToSend[4] = (byte)points[1];
        bytesToSend[5] = (byte)points[2];
        bytesToSend[6] = (byte)points[3];
        bytesToSend[7] = (byte)points[4];
        bytesToSend[8] = (byte)points[5];
        _serialPort.Write(bytesToSend, 0, 9);
        Console.WriteLine("Sent: D" + bytesToSend[1] + "-" + bytesToSend[2]);
    }

Notes learnt from hooking all this up

  • CAN Filters work, but you can still see messages
    On the CAN controller you can set filters to only allow specific messages through. It actually turns out that, regardless of the filter being set, if you constantly poll either buffer for a message that, if there has been a message sent, it will be available and can be received from the network. Filters simply prevent the message from triggering the 'message waiting in buffer' interrupts. So, the best bet is that you check for a message in a buffer first rather than simply constantly attempt to receive messages. Let the CAN IC tell you that a message is waiting!
  • You can easily spam a CAN network
    Sending too many messages over the network will delay or prevent transmission of other nodes' messages. One node can send too many messages over the network preventing another from successfully getting messages into a third nodes' buffers. It's all down to how quickly the third node can receive the messages. If both receiving buffers are full then the message will fly on past and not be seen.
  • Inserting and removing ICs
    PLEASE only do this when you need to and do this with care! Lever them out from both ends and DO NOT just use your fingers. Insert them evenly as well. It's too damn easy to destroy these delicate and expensive components.

...next will be a node to control the thottle, isolated blocks and point-servos.

28Sep/110

Ruby on Rails – Disturbing Errors

Excuse my efforts in going slightly off topic in an attempt to help out anyone else who may have struck this issue.
I'd been given a good challenge by a friend: mimic an install of ruby on rails on a local VM. I'd never worked with ruby before and google'd for a tutorial. Before I knew it I had Turnkey Linux's Ruby Appliance up and running on Oracle's Virtual Box on 64-Bit Windows 7.

I wont go in to deployment steps here... it's all covered on the web. The basic concepts were to point an apache virtual host at a new directory in /var/www and then browse to the application. To my delight it started, throwing an error. I hadn't even configured a thing yet, so how could I expect it to run?

After configuring the Facebook connector and the database connection the site came up and I could log in and muck around. It wasn't long until I hit the following error. It was only on a specific data class that extended the ActiveRecord interface and only when checking if the data was valid or trying to save it. I could happily create the class and change attributes...

Wrong number of arguments (1 of 0)

What the hell... I hadn't seen ruby exceptions before, so I started to dig. The RAILS_ENV environment variable was set to 'development' and so there was a 'development.log' file sitting in the 'log' directory inside the application. This provided a stack trace and a few other bits and pieces.

After a lot of code commenting and uncommenting I narrowed it down to the 'save' method of the specific data item class. This was inherited from the ActiveRecord::base class and I had no say in what it did or how it did it. I did know that it was expecting no arguments, so that's where the '0' came from in the error, but I was also not passing it any arguments, so where the hell did the '1' come from!?!?!

I then tried the 'valid?' call as well... same result.

I chose to override the 'save' function in the model file by using a pass through... all of a sudden the code continued (after hours of exceptions) and the next related table row creation had a null foreign key for the item that should have been saved. It turns out that by creating a pass through I bypassed the entire 'save' method and therefore didn't attempt to write anything to the db. As soon as I tried to use the base 'save'I got the same error but now in the model file.

I was stuck... so I tried to bring up the debugger via the mini WEBrick server. It, unfortunately, didn't give me any more information than what the development.log file had provided.

Next I tried downgrading and upgrading ruby and rails. I was impressed with the 'gem' system... but nothing helped.

The final step was to google more and more... I had already tried gutting my model implementation (there was extra code in the for other tasks) but none of this changed the inability to call 'save'. All of a sudden one google search came across someone mentioning something about a 'reserved word'.... I google'd further.

Reserved Words

It seems that Ruby has a few words that you MUST NOT GO NEAR if you want your code to work. The main issue is that, when you do, you'll have no idea what you have done as the error message that results has NOTHING TO DO WITH the actual problem. It turns out that after browsing to a post by someone with a similar problem I saw the comment stating that they had used the word 'notify' for a column name.

... so had the original developer of the code I had worked on ...

There was no response to the post that pointed this out in the previous link, so that either means the original author was too ashamed and it was the correct fix, or they found the real fix and never got back to this.

Either way, I corrected my database by changing 'notify' to 'notify_opp' and ... fkn presto ... 10 hours of debugging resulted in a SUCCESS!

Further information on these magical reserved creatures

Ok, the fun part... in hind-sight I've done a little research and there are sites indicating what 'words' are 'reserved' for ruby. Unfortunately, not all of them say that 'notify' is!

References:

Conclusion

Be bloody careful with errors from ruby that don't make sense and don't spend too long digging in to a dead end.
Be diligent with your google searches too.

I spent a lot of time debugging this and I hope this post helps out others in the same scenario.

27May/11175

Commandos – Behind Enemy Lines Resolution Fix

Note: Both Commandos - Behind Enemy Lines (BEL) and Commandos - Beyond The Call Of Duty (BTCOD) (Plus a crapload of other versions, see below!) are supported!

STEAM versions are not supported. They're encrypted and this patch cannot be applied.

You can also use this patch to carry out the Locale hack described here.

Skip straight to the bottom of this post if you just want the application!

An Introduction to the game

A friend and I recently stumbled across the Commandos Ammo Pack ('Behind Enemy Lines' and it's expansion pack 'Beyond The Call Of Duty') at Good Old Games.com and we just couldn't resist downloading it. It did cost US$5.99, but that's absolutely chicken feed when compared to the awesomness of this game.

It's an old game, created in 1999, and could handle resolutions up to 1024x768 for single player; but for multiplayer the resolution was fixed at 640x480. I google'd around a little and saw that one person on a forum had created the Mysoft - Commandos Loader v0.8 which seemed to hoist the application in to memory and then hacked around with variables. I wasn't sure this was the best method and had the main Commandos EXE in IDA Free before too long.

I'd read that it used DirectDraw/3D and started searching for the appropriate functions. Before long I had the AdjustWindowRectEx, DirectDrawCreate and other functions under the microscope. I then opened the EXE up in OllyDbg and started slapping breakpoints everywhere. Before long I'd noticed the pattern:

push    10h
push    1E0h
push    280h

This seemed to be popping up everywhere and co-incidently 280h x 1E0h = 640x480 pixels... the default for the game. I then found code that set the resolution based on those shown in the video settings menu and then the game was doing what I told it.

commandos-512-384 commandos-1024-768 commandos-1680-quit

After a little more google'ing I found that someone else had already gotten to this point. Ferdinand had the hacks there to change the resolutions, but did not have the final piece to ensure that re-loading the game brought the resolution back.

After a little more disassembling I found that the resolution was being loaded at startup from a file on disk... this turned out to be "COMANDOS.CFG" in My Documents. The exact line from the config was:

.SIZE [ .INITSIZE 2 ]

From the config it was obvious that the game was using 'index 2' from the resolution list. Of course, it also meant that it knew how to decipher that '2' to a real resolution from the list in the code. After a little forward and reverse searching from the file loading to the resolution setting I came to the switch statement in the code which had the values hard-coded. I now had 3 points per resolution setting in the code that needed to be changed if I was to write my own utility.

Commandos Resolution Hack Alpha 1

I hadn't liked anything that was already available to do this and so I chose to wrote my own. The application loaded the EXE into memory, read out the resolutions stored, audited that they were all in sync and then allowed the user to select a new resolution per slot. If the resolutions weren't the same then you would get a warning, but this would have been corrected on the next resolution hack.

commandos-file-selection commandos-resolution-hack

This worked fine, but I found it pointless to specify four separate resolutions when you only really needed to specify one. I was more in for the challenge of allowing all four to change, rather than the practicality.

Testing and more bugs

So, it all worked... I then closed the game, re-opened it and found that it loaded up at 1024x768 instead of the 1680x1050 resolution I had set. I closed it again and looked at the configuration file; the setting had indeed changed back to '2' instead of '4'... but I hadn't touched it! I then loaded the game again and it was back to 1680x1050... closing and opening brought it back to 1024x768... the flip-flop continued.

So... what did this seem to mean? I had overlooked the storage of the resolution configuration. If the indexed resolution was read from the file, then it would have to be written back as an index. I went back in to the disassembler and searched for anything relating to writing 'SIZE' into the configuration file. The logic in the game was then obvious: it carried out an "if less than X but greater than Y" across the 'stock' resolutions and then stored the relevant index. This complicated my approach and confirmed that I really should only bother adjusting the 'fourth' resolution in the list. There was no point allowing the user to re-order the resolutions as this code then wouldn't work. The fourth resolution would need to be the only one customisable and the value must be greater than the third resolution at 800x600.

Commandos Resolution Hack Alpha 2

So, with this in mind, I wound the application down to just editing one resolution that had to be higher than 800x600. This meant that the internal configuration saving would work and the user would just have to select the final resolution in the list. Note that this only affected single player... the multi player resolution would still be a separate setting.

commandos-final-version

Everything was now working great... except the drawing issues above 1024 resolutions...

Drawing bugs

So, the game obviously wasn't meant to be played with a horizontal resolution greater than 1024 pixels; rendering artifacts started appearing when higher resolutions were used. Fortunately, the game is quite dynamic when it comes to rendering the screen and menus. Trawling through the code I could see that it was trying to find a BMP that matched the resolution, even using the resolution as the file name. Ferdinand had also worked this out and even has a list of assets downloadable for higher resolutions.

Commandos stores all of its graphical assets in a file named 'WARGAME.DIR' and references this when the game loads. I started looking into modifying this file via my program but realised it would be wuite a task to decipher and re-pack. Fortunately, some clever fellows across the world have created the necessary tools to extract these files. DirExtractor allows us to expand the DIR files and, once in the Commandos directory, the game itself will actually use the extracted versions if you remove the WARGAME.DIR file. This then meant that I could use my app to call DirExtractor, extract the files and then modify them where required to ease the majority of graphical glitches.

commandos-1680-resolutions commandos-1680-menu commandos-1680-loading

Required Graphical Modifications

  • A single change to the fourth resolution string in DATOS\MISIONES\GLOBAL.STR to whatever the custom resolution has been set to. This can be found by searching for the token OVI4 and then replacing the string following.
  • The creation of the menu background. This has to be a Bitmap file named MENU####.BMP where the #### is the horizontal resolution of the screen.
  • Build a WAD file for the game-play interface. This WAD is named via the resolution (i.e. 1024X768.WAD) and contains values that tell the game how to render the interface. I created a one-size-fits-all file that ensures that the graphics are wide/long enough for the screen resolutions (max 1920x1920) and the application simply renames the file and ensures that it is available to the game.

A final glitch

There was only one drawing issue remaining after this... any resolution over 1400 meant that maps thinner than the overall monitor width would not re-draw the areas that they could not cover. You can see this in the following screen shots.

commandos-bad-map

Fortunately you can press '+' and '-' in-game to zoom in and out. What this meant was that the user could stop the map from clipping by adjusting the zoom. I took this as a suitable workaround at the time.

commandos-1 commandos-1

Smeared resolution?

It seems that some laptop resolutions are a little out-of-whack. I previously had a Samsung Ultranote with resolution of 1366x768, but windows reported that it could also run at a resolution of 1360x768.

It turns out that if you choose the wrong one (although it may seem the right one if that's your current windows resolution) that you'll get the following display:

commandos-smeared-menu commandos-smeared-res

Simple fix: Try another resolution... it'll work eventually!

And you thought it was going to be this easy?

So, I had the app working great for the modification of the GOG version of the EXE. It turns out that there are many more versions in the wild. It also gets worse as there is another patch here that fixes up a whole lot of other issues in the game. Fortunately it seems that the EXE from the above link is the exact same as the EXE from GOG? Either way, I've tested it and my hack works on it perfectly.

Due to the above, I started a list of EXEs for the game and then began deciphering them (as I knew the basic items to change) to allow my application to work with them.

Items in bold are those that come from GOG.com.

Game Version File Size Supported
Commandos - BEL Demo 2,452,992 bytes Works..
Commandos - BEL German v1.0 2,469,376 bytes Works, Asks for CD (Language issues, I'd imagine)
Commandos - BEL Spanish ? 2,469,376 bytes Works?
Commandos - BEL Spanish v1.0 2,469,888 bytes Works?
Commandos - BTCOD Spanish ? 2,696,600 bytes Works?
'Sold Out' v1.1 2,479,104 bytes Works fine. Speed issues with game (known issue)
GOG.com Ammo Pack v1.1 3,715,072 bytes Works perfectly!
Commandos Ultimate Fix v1.1 3,715,072 bytes Works perfectly!
Commandos - Beyond the call of duty ? 3,133,440 bytes Works perfectly!
Commandos - BEL Russian ? 2,470,400 bytes Works fine. Speed issues with game (known issue)
Commandos - BTCOD Russian ? 2,968,576 bytes Works fine. Speed issues with game (known issue)
Commandos - SINP v2.3 2,482,688 bytes Works perfectly.
Commandos - SINP Chinese v2.3 2,313,216 bytes Does not write out chosen resolution to configuration file. You should only change the resolution via this hack!

My application was then built to determine the EXE via file size (of course, this isn't fool-proof) and then hack appropriately.

Commandos - Beyond The Call Of Duty

So, this was just an update to the main game... turns out the code was slightly different, but still workable with my current program structure. After about 6 hours debugging all was working! Enjoy!

commandos-btcod-1 commandos-btcod-1

Commandos Hack v1.2

The final release contains the following features:

  • Support for all versions known above. (Note that the Demo and German versions don't actually save their configs to disk; we therefore hack the initial startup resolution.)
  • Resolution list options based on monitor capabilities.
  • Both EXE and WARGAME.DIR are backed up.
  • A one-size-fits-all menu background has been included.
  • German file encoding is preserved when the text is changed.
  • Resolution menu text shows the selected resolution.
  • Compatibility mode now checked to allow proper resolution and disable scaling.
  • The Locale hack from here has also been added.

commandos-hack-final

Note: You need .NET Framework 4.0 to run this, download it here.

Download the application here.
And you can also download the .NET source code here.

Meanwhile, if you got a lot out of this app, then feel free to donate!








And that's a wrap... go use every pixel your monitor can produce and play the game!

16May/110

Creating two more MAME Controllers

I've since realised that, after building the original two MAME Arcade Controllers, I've needed two more to really re-live the old classic games. I suppose one more would've been OK for games like Rampage, but TMNT, Simpsons Arcade and Sunset Riders need all four players... so... the following lists the relevant information to build controllers 3 and 4.

Sunset Riders

It turns out F12 saves snapshots of MAME games... so I couldn't resist...

Sunset Riders Sunset Riders Sunset Riders
0005 Sunset Riders Sunset Riders

Key Mapping

Below lists the mapping for the two new controllers to be created. I went ahead and used the default keys from MAME plus whatever else was free...

NOTE: Out-of-the-box MAME does not specify buttons 4,5,6 for Players 3 and 4! You can set these very easily in the 'default' config by using the basic MAME GUI (run mame.exe without any command line options)...

Button Player 3 Pins Player 4 Pins
Joy UP I 6, 22 Numpad 8 8, 22
Joy LEFT J 7, 21 Numpad 4 4, 23
Joy DOWN K 6, 21 Numpad 2 8, 21
Joy RIGHT L 5, 21 Numpad 6 14, 23
Button 1 R-CTRL 13, 20 Numpad 0 8, 24
Button 2 R-SHIFT 10, 20 Numpad . 14, 24
Button 3 ENTER 1, 20 Numpad ENTER 12, 21
Button 4 / 3, 26 Numpad + 12, 22
Button 5 . 5, 20 Numpad - 12, 26
Button 6 , 6, 20 Numpad * 12, 20
Player Button 3 16, 19 4 11, 19
Insert Coin 7 7, 19 8 6, 19

Building the other two controllers

I bought cheaper controls for these two... but to no real benefit. I did save AUD$10, but I don't really like the button microswitches. The buttons aren't as tall and they do look cheaper. They're also a lot less 'clickier'. On the positive side: the short-stick joysticks feel nicer during game-play and are a lot more sensitive.

Player 3 and 4 Player 3 and 4 White vs. Black
LITEON Board?

The build of these two followed the exact same path as the previous two (see the previous post) using cheap Microsoft Keyboards. It turns out that they were now a different colour (White vs. Black) and I decided to replace my current keyboards on other computers with the new ones. I therefore used a standard black MS Keyboard for Player 3 and a 'Microsoft Comfort Curve' for Player 4.

The MS Comfort Curve had a completely different matrix and the circuit board inside was much easier to solder to. Either way, the process was still the same in determining the pins and then wiring everything up. Testing was made a lot easier as well as I found the below testing applications to work out exactly what I'd messed up :)

Player 3 Player 4 All 4 lined up

After running out of solder and then later butane I got them together and we gave them a go. All went well until Players 1 and 4 hit magical key combinations... It seems that, although the keyboards are separate devices, you can still produce an ALT-ENTER or CTRL-ALT-DEL by having each player pressing each of the keys separately. This has since been remedied in the final section of this post!

Testing your new creation

I found out that Windows 7 (and possibly Windows Vista?) but not Windows XP has the mskey.exe keyboard test application (I have since found out that you can Download the Intellisense Software here which contains this application.) It came in very handy until I plugged 12v into a USB hub and cooked the HDD in my TV-pc (luckily only the HD, the PSU managed to protect everything else!) and lost my Windows 7 install. For Windows XP systems (of which all my computers are now back to) I downloaded and used the PassMark KeyboardTest application. At first, the Google search result made the application look like a typing tutor, but after downloading I realised it was exactly what I needed.

Microsoft Keyboard Diagnostics PassMark KeyboardTest

MAME Configuration

So, as previously mentioned, MAME doesn't specify Buttons 4,5,6 for Players 3 and 4 by default. I suppose there aren't many games that have more than 3 buttons for all four players? I can't think of any... but my memory is quite hazy as to what 4-player games Combat Zone had back at Tuggeranong Hyperdome.

Either way, we've specified them above and we also need to make sure they map to the correct player/keys in MAME. Here's a snippet for Player 4 in my default.cfg:

<input>
 <port type="P4_JOYSTICKRIGHT_UP">
  <newseq type="standard">KEYCODE_6PAD KEYCODE_8PAD</newseq>
 </port>
 <port type="P4_JOYSTICKRIGHT_DOWN">
  <newseq type="standard">KEYCODE_2PAD KEYCODE_6PAD</newseq>
 </port>
 <port type="P4_JOYSTICKLEFT_UP">
  <newseq type="standard">KEYCODE_4PAD KEYCODE_8PAD</newseq>
 </port>
 <port type="P4_JOYSTICKLEFT_DOWN">
  <newseq type="standard">KEYCODE_2PAD KEYCODE_4PAD</newseq>
 </port>
 <port type="P4_BUTTON4">
  <newseq type="standard">KEYCODE_PLUSPAD</newseq>
 </port>
 <port type="P4_BUTTON5">
  <newseq type="standard">KEYCODE_MINUSPAD</newseq>
 </port>
 <port type="P4_BUTTON6">
  <newseq type="standard">KEYCODE_ASTERISK</newseq>
 </port>
</input>

As you can see, the default.cfg only adds in controls we have overridden... the others must be hard-coded into mame.exe. Drop the configuration file in to your mame/cfg directory (note that you may want to check your file first as you could have other customisations!) and then you'll have the appropriate settings ready to go!

Diagonals and 4-Way Joysticks

Right, I couldn't work out why I couldn't get 'Steve' in 'Sunset Riders' to shoot on the diagonal... usually this was carried out by holding down the Up and Left (or Right) keys on the Keyboard, but it wasn't working. I scoured the internet and initially thought it was an issue with the way I'd wired up the joysticks.

Sunset Riders Sunset Riders Sunset Riders

It turns out that MAME has added in the ability to assign a separate key for the diagonal movements. This is great if you have an 8-way Joystick with actual individual microswitches on the diagonals, but I have a 4-way and I cannot, without logic circuits, wire up another key to activate when two of the direction microswitches are closed. I did actually start thinking of logical OR/AND gates to fire the specific keys when the direction goes diagonal, but this would require re-work on the sealed controllers. :)

mame-config

So, I went in to MAME and then mashed the keyboard on the configuration screen. It turns out that you can assign multiple keys to a direction. This then made life easy... I would make Player 1's "Up/Left" key 'Up' + 'Left' instead of the 'I' key on the keyboard.

After testing this new configuration I realised it wasn't actually MAME that was the problem! It turns out that there was an actual physical issue with the initial two joysticks I bought when compared to the second two...

Arcade Joystick differences

So, the diagonals worked on the new joysticks I bought... they seemed to be a lot happier at pressing two directions at once. Unfortunately, the original joysticks only really wanted to go one way at once... After a little investigation it turns out that the plastic spacer/washer that slides onto the joystick stem was not wide enough (circumference-wise) to actually activate two microswitches at once. I'd also misplaced the original 'extra' components from the intial build and therefore didn't know if wider spacers existed? Either way, I had to make the spacers fatter to ensure that diagonal movements could be easily made.

In the end I cheated and used duct tape around the spacer at the bottom of the stick to make it's circumference wider. This meant it was more keen to press two microswitches at once and the world was a happier place!

Spacer on original Joysticks Spacer Bigger spacer

The sticky-tape-fix was then tested for Player 1 and worked. I then did the same for Player 2 but had the controller back open in seconds after playing Bomberman as it turns out it was now too happy to go diagonal instead of 4-Way. Removing just over one layer of tape from the spacer on the joystick balanced out the circumference vs. microswitch equation and made the joystick much more responsive. Your mileage will vary with how much tape is required based on the sensitivity of the microswitches and how much travel/space you have between them and the center stick.

The verdict: Windows short-cut keys kill gameplay

The next step was to reconfigure any keys on the controllers to not use any modifier key (CTRL,ALT,SHIFT,ENTER) so as to not activate shortcuts during gameplay. When Player 1 and Player 4 pressed certain buttons (specifically ALT-ENTER) the MAME window would restore and, of course, it was possible that all four players could do a perfectly orchestrated CTRL-ALT-DEL and throw Windows XP to the Task Manager.

To do this I had to re-solder buttons on Controllers 1, 2, 3 and 4 to the nearest free keyboard button. Fortunately the Numpad had more than enough nearby keys free and there are still letters of the alphabet unused near Player 1's keys. The layout of buttons on the keyboard did not have to make sense to a human user and so any free buttons were used if the wiring was easier.

The Final Re-Mapping

So, in the end I re-mapped the modifier keys and also removed the duplicates between Players 2 and 3. Edits are in bold...

NOTE: You must UNMAP 'T' from the 'Other Controls' area otherwise when Player 1 hits Button 2 you may receive a 'Tilt' message and game restart.

Button Player 1 Pins Player 2 Pins
Joy UP UP 12, 24 R 11, 22
Joy LEFT LEFT 12, 26 D 16, 21
Joy DOWN DOWN 4, 26 F 11, 21
Joy RIGHT RIGHT 8, 26 G 11, 24
Button 1 E 16, 22 A 18, 21
Button 2 T 11, 23 S 17, 24
Button 3 C 16, 20 Q 18, 22
Button 4 V 11, 20 W 17, 23
Button 5 Z 18, 20 U 7, 22
Button 6 X 17, 20 O 5, 22
Player Button 1 18, 19 2 17, 19
Insert Coin 5 11, 25 6 7, 25
Button Player 3 Pins Player 4 Pins
Joy UP I 6, 22 Numpad 8 8, 22
Joy LEFT K 6, 21 Numpad 4 4, 23
Joy DOWN J 7, 21 Numpad 2 8, 21
Joy RIGHT L 5, 21 Numpad 6 14, 23
Button 1 B 11, 26 Numpad 0 8, 24
Button 2 N 7, 26 Numpad . 14, 24
Button 3 M 7, 20 Numpad / 8, 20
Button 4 / 3, 26 Numpad + 12, 22
Button 5 . 5, 20 Numpad - 12, 26
Button 6 , 6, 20 Numpad * 12, 20
Player Button 3 16, 19 4 11, 19
Insert Coin 7 7, 19 8 6, 19
Download the configuration file here.
This configuration overwrites the diagonals for Player 1 and adds/corrects all of the other mappings listed above.

And this is now to be tested in gameplay... I've done the re-wiring and have tested individually but have done no actual thrashing with MAME.

The End

And that's that... it was enough fun playing with Player 1 and 4 making sure not to mash the same buttons at the same time... but it should be a lot better now once I organise the beer and the people to play... will see how we go!

5Mar/1113

Creating your own MAME Arcade Controllers

Right, now for something completely different... The following article goes out to all those 80s kids who remember that familiar clicking sound of real mechanical microswitches. We all spent our pocketmoney on Bubble Bobble, Snow Bros, Rampage, Street Fighter; but now you don't have to. Just install MAME and build the following controllers...

Here's the photo album of the construction of the controllers...

Note, before starting this, you should really just skip the keyboard hacking and buy one of these.
Also, for a LOT of information on doing this, visit Arcade Controls.

Initial thinking...

All peripherals for a computer need to connect via an I/O port. Although there are quite a few available on most modern machines, the one we will use is USB. I've chosen this port because they will be around for decades and the old style Serial, LPT and Gameports have pretty much already been phased out.

Now, there are many options once we choose USB:

From the above, the cheapest method is to rip open a keyboard. Officeworks here in Australia was selling the Microsoft Wired Keyboard 600 for AUD$11.97 and I couldn't resist.

MS USB Keyboard MS USB Keyboard

One big question here was cabling. Once I'd hacked a keyboard apart and wired up one controller.. would I then use other keys from that first keyboard and run a cable to the second controller? I looked at the price of data cabling and realised it was cheaper to buy another USB keyboard than it was to rig up a sophisticated central box that both controllers ran to.

I therefore bought 2 USB keyboards.

Now, there is an issue here... I will be 'emulating' keyboard buttons presses when the joystick buttons are pressed. Since once computer can accept multiple different keypresses at once, this will work, but it cannot distinguish multiple presses of the same letter/key from different keyboards (i.e. if the 'A' key is pressed from two different devices.) This means that the second controller will have to be mapped to a different set of keys... pretty obvious, yes?

Update/Note that buttons 5 and 6 for Player 2 aren't set by default... I've set these to I and K as they were free... for some reasons I didn't check Player 3 and 4 controls first and this is a direct conflict with Player 3 directional keys! So, you can use the below but be careful... in my this blog post I've remapped them to U and O respectively.

Button Keymapping 1 Pins Keymapping 2 Pins
Joy UP UP 12, 24 R 11, 22
Joy LEFT LEFT 12, 26 D 16, 21
Joy DOWN DOWN 4, 26 F 11, 21
Joy RIGHT RIGHT 8, 26 G 11, 24
Button 1 L-CTRL 13, 25 A 18, 21
Button 2 L-ALT 9, 24 S 17, 24
Button 3 SPACE 1, 26 Q 18, 22
Button 4 L-SHIFT 10, 23 W 17, 23
Button 5 Z 18, 20 I 6, 22
Button 6 X 17, 20 K 6, 21
Player Button 1 18, 19 2 17, 19
Insert Coin 5 11, 25 6 7, 25

Hardware

So, from the above, you'll need 2 USB keyboards for the PC interface. You'll then need the actual arcade joysticks and controls. Below is the kit I found recently on eBay for around AUD$50. I'm sure you could get it cheaper; my purchase was on a very large impulse and I had a few regrets afterwards as there were better joysticks on offer for the same price.

Arcade Controller Kit Player 1 and Player 2 Joysticks!

The buttons themselves are exactly what you'd find in real arcade machines... the microswitches make their motion feel perfect as well.

Arcade Button Arcade Button + Microswitch

So, with all the components, it's now time to get it connected together. Start by ripping open the keyboard...

Decoding a USB Keyboard

Keyboards are built on a matrix. Just like the LED project previously with the Arduino, each button on a keyboard is connected to one row and one column of the circuit board inside. The circuit board will scan each row and column hundreds of times a second to check which buttons are pressed and then report this back to the computer.

Dismantling MS USB Keyboard Opened USB Keyboard Button layering
Controller + Keyboard Matrix Splitting Matrix

As you can see above, the controller inside the keyboard is not soldered to the matrix. This is great news as I was expecting to have had to solder to fiddly little wires instead of the nice connection pads that you can see.

Firstly I pulled out the multimeter and started decoding the keys I'd need to use. To do this I put one probe on the pad circle of the key and I then ran the other probe across the connection to see which wire it connected to... I then recorded all of my findings:

Multimeter Deciphered key codes

The data I gathered is in the table above. Each key has two pins and these map from left to right on the controller connection seen above. I have only decoded the keys that I need, but it's easy enough to work out the rest when you have an opened keyboard in front of you.

I then began to wire up the controllers. One the first attempt, I thought I would solder short wires to the controller and then use these for connection points. This worked OK until I was up to the final wires. It seems that constant moving around of the controller puts pressure on the soldered joints and I ripped many of the tracks off. I then had to get my tiny copper wire and re-solder the tracks as the pads were gone. This became quite tedious, but I learnt my lesson very quickly.

Soldering to controller Broke a track... Copper to the rescue Player 1 controller wired

I got enough of the wires connected for the first joystick and then quickly connected it to test. I then managed to play a quick game of Bubble Bobble. Impressive! You can see below my initial prototype housing for the controls. The washing basket was the worst idea on earth... but it was a cheap mistake.

Mounted on cardboard for test Failed washing basket prototype Playing Bubble Bobble

Building real housing

I ended up at Bunnings looking for ideas for a box to house the controls. There was nothing off-the-shelf, so I opted for enough wood to make a deep enough box for the controls. This was a pretty simple and ugly construction... but it did the job.

Construction of actual housing Marking out control spacing Construction of actual housing
Building Player 2 Power drill + Hole saw So much quicker than manual drill
Wiring up the controls Completely wired up Player 1 controller ready

Finished products

After testing, I put backing panels on both of the controllers.

Controller done Player 2 finished

Each controller can only ever be used as Player 1 and Player 2 respectively as they are mapped to specific keys on the keyboard. Of course, you could change the configuration of MAME, but that would be tedious... Although sometimes on games it'd be nice to be able to play as one of the other 2 players (3 or 4) ... but I'll create two more controllers for that.

Future plans

I want to build two more of these... 'The Simpsons' arcade and 'Teenage Mutant Ninja Turtles' require 4 players for the most amount of fun, and 'Rampage' needs all three creatures bashing up buildings.

11Feb/111

Lighting a Japanese Temple

Actual Temple Kit

I'd decided it was time to light the temple after building the Torii for the entrance. This temple was the Tomytec Japanese Temple A (Main Building) and is still available for purchase from most Japanese online hobby retailers.

I've slapped LEDs in buildings before, but this time I also wanted to add lanterns to the front of the shop. I'd made the lanterns before, as in my previous attempts of creating the Torii, but I was to make a few changes this time as I wasn't totally impressed with the previous outcome.

Creating the lanterns

There was a slight change this time to creating the lanterns... instead of cutting them and sliding them over the LEDs, I shaved them down to fit and inserted them into the center of the tubing. This all worked well, but you must be careful when shaving down the LEDs as you can destroy them quite easily. To shave the LEDs, I held them in pliers in one hand and filed away with my pocket knife. It was pretty obvious to feel when you were no longer filing away at plastic and, unfortunately, this was usually the demise of the LED.

Shaving down LEDs LED inside plastic tubing

Mounting the lanterns

I used the same copper winding wire that I always do and bent it into a rectangular shape to fit the roof of the temple. I then started soldering the lanterns in place.

Lantern on wire Using wire to frame lanterns in place

I then pulled out the trusty Selleys Aquadere and, using random aligator clips found on the bench, glued the lanterns in place.

Gluing down lanterns in place Aligator clips do wonders

I also put two standard 3mm white LEDs in the center of the ceiling for building lights.

Central building lights

The finished product

After the glue had dried, I tested all the LEDs and found that I'd broken the front-left lantern. This was 24hours after starting the project and frustrating. I quickly removed it from the temple and filed another LED down. I left it dry again, overnight, after testing, gluing and testing again.

Finally, yesterday, I was able to hook it up to my Arduino LED Controller. It worked perfectly and I took the opportunity to test my night-time photography skills.

Finished job View past the city And the Torii
A different view Night time Top Platform

Now to settle the landscape around it.

18Jan/110

An attempt to simulate Acceleration and Braking

In my previous post, I'd managed to get my Densha-De-Go Dreamcast controller hooked up to my Arduino Mega. Now although this now meant that I had a great way to control my model railroad, it also meant I had to work out how to code a throttle and a brake lever.

The rules

After a few hours of plotting, I had decided on the following system. It involves a 6-position throttle and an 11-position brake. Each 'position' is to have a 'max speed' and 'speed adjust' associated with it.

Notes

  • It is to be assumed that if the brake is on, the throttle is automatically disabled
  • MAX is 255 on the PWM Throttle (or max voltage from supply)
  • At Throttle 0, the train is neither powering nor braking; we will simply slowly decrement the speed
  • There is no feedback to know how fast the train is currently travelling
  • There are multiple emergency brake points on the throttle, but we don't care about them.

The next table shows my acceleration and braking deltas. This will be a simple addition and subtraction on the current speed.

Lever position Max Speed Speed Adjustment
Emergency Full Instant Stop
Emergency 5 -50
Emergency 4 -30
Emergency 3 -25
Emergency 2 -20
Emergency 1 -10
Brake 9 -2.4
Brake 8 -1.8
Brake 7 -1.2
Brake 6 -0.8
Brake 5 -0.4
Brake 4 -0.2
Brake 3 -0.1
Brake 2 -0.05
Brake 1 -0.025
Throttle 0 0 0.00
Throttle 1 55 +0.25
Throttle 2 75 +0.5
Throttle 3 90 +1
Throttle 4 100 +1.75
Throttle 5 120 +2.5

And now a better way to represent it.

brake-accel-graph

The code

The table above translates to code quite easily... the goal is to have the lever position values coded into an array and then just select the correct entry. Once determined, the main code loop can then determine how to adjust the current voltage output to the rails.

const int throttle_positions = 21;
const int throttle_absolute_maximum_speed = 255;
const int throttle_minimum_speed = 20;
int current_throttle_position = -1; // 0 is EM FULL. Lever must be moved to EM FULL to begin.
float current_speed = 0;
float target_speed = 0;
int throttle_max_speed[throttle_positions] =  {
	0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, //brake positions and Throttle 0
	0, 55, 75, 90, 100, 120
};
float throttle_delta[throttle_positions] = 	{
	0.025, 0.05, 0.1, 0.2, 0.4, 0.8, 1.2, 1.8, 2.4, 10, 20, 25, 30, 50, 9999, //brake
	0.00 /*coast*/, 0.25, 0.5, 1, 1.75, 2.5
};

We then need to determine the current throttle position. We will make it that, at the start of code execution, the train should not move until the throttle has been reset to EM Full and the throttle at '0'.

#define BRAKE_MASK 0xf0
#define BRAKE_SHIFT 4
#define ACCEL_MASK 0x07
void read_throttle_position() {
	int accel = packet.data[6] & ACCEL_MASK;
	int brake = (int)((packet.data[7] & BRAKE_MASK) >> BRAKE_SHIFT);
	
	if (current_throttle_position == -1) {
		//check that we have EM FULL and Neutral
		if (brake  == 15 && accel  == 1) {
			//set the initial '0' (EM FULL) position.
			current_throttle_position = 0; 
			lcd.clear();
		}
	} else {
		if (brake != 1) { //1 == "BRAKE 1", if it's higher, we're braking.
			if (brake > 0) current_throttle_position = brake;
		} else { //BRAKE == 1 and then we check the throttle
			//we're accelerating.
			if ((accel + 15) < 22) current_throttle_position = accel + 15;
		}
	}
}

And now the main game loop needs to determine the current lever locations and then choose the appropriate action:

void update_speed() {
		digitalWrite(13, LOW);
	//make sure we are allowed to go.
	if (current_throttle_position >= 0) {
		if (current_speed > throttle_max_speed[current_throttle_position - 1]) {
			current_speed -= throttle_delta[current_throttle_position - 1];
			
			//braking... don't go negative.	
			if (current_speed < throttle_minimum_speed) 
				current_speed = 0; 
				
		} else if (current_speed < throttle_max_speed[current_throttle_position - 1]) {
		
			//accelerating, so start from minimum speed.	
			if (current_speed < throttle_minimum_speed) 
				current_speed = throttle_minimum_speed;
				
			current_speed += throttle_delta[current_throttle_position - 1];
			
			//don't go faster than current throttle max setting.
			if (current_speed > throttle_max_speed[current_throttle_position - 1])
				current_speed = throttle_max_speed[current_throttle_position - 1];
		}

		//set light if we have met max speed for throttle.
		if (current_speed == throttle_max_speed[current_throttle_position - 1]) 
			digitalWrite(13, HIGH);
			
		//output speed to railway.
		analogWrite(7, current_speed);		
	} else {
		//flash the LED to alert user to reset controls.
		delay(200); //delay a little to flash the LED
		digitalWrite(13, HIGH);
		delay(200);
	}
}

There's also some code in the main loop to update the 16x2 LCD I've hooked up. Since we need to reset the controller when we start, it'll tell you to do so and afterwards will tell you the current throttle/brake position, current speed and speed delta.

Download the source code here.
Note that this does not include the full arduino-maple code. Download that here. You will also need the LiquidCrystal library from the Arduino site.

Action shots

Shown is the controller in certain positions. Note that the 'Throttle' positions may show a negative speed delta; this just means that they are no longer accelerating.

DSC03765 DSC03766 DSC03770

DSC03775 DSC03776

17Jan/116

Arduino + Dreamcast Densha-De-Go Controller

I've had Densha-De-Go and the controller for the Dreamcast for a while now... I'd even invested a few hours in playing the actual game, but the accuracy required is crazy. Supposedly in Japan it's that realistic that even real train drivers have a hard time getting it spot on :)

Anyway, I'd wanted to get this going for train control for a while after managing to make a Wii Nunchuck control my trains.

DSC03710 DSC03712 DSC03715
DSC03707 DSC03698

You can find more information on the controller at SEGA Retro, SEGAGAGA Domain, Wikipedia, Play Asia and genki Video Games.

Previous attempts

Quite a few months ago I spent a few hours with my Densha-De-Go controller and my Arduino Mega in an attempt to get them to communicate. The goal was to be able to read the controller and then use it to control my model railroad. Unfortunately, I was doing this blind, running off information from Marcus Comstedt's 'Dreamcast Programming' site; specifically his breakdown of the Maple Bus Protocol.

I ended up with nothing working and sent out a few pleas for help on the Arduino forum and to Marcus himself. I received information regarding the fact that it should work (the Arduino had the horsepower), but I would have to ensure the timing was intricate and that the Arduino was always ready to receive data.

I then posted to the Arduino Forum for help. A few months later WzDD came to my rescue with word that he had successfully made the Arduino communicate with a Dreamcast Controller. I hadn't had much time up until now to work on this, and WzDD hadn't done any further work on it, so I took it that he just wanted to prove the concept and that I'd have to get off my backside to make things progress further. In the end, I did want to get my controller functioning.

I had previously had everything I needed to test this again, so I gave it another go. My previous setup was a cut-in-half Dreamcast controller extension cable and my Densha-De-Go controller. The extension cable meant I didn't have to hack around with the actual controller or port imitation.

Getting set up...

Due to the requirement to use assembler for the actual low-level controller interactions, we cannot use the Arduino IDE and must use WinAVR (or just avrdude on Linux.) Download this from here (select the latest version) and install to the default directory.

You then need to download the arduino-maple source code and extract somewhere locally. I've put the folder on C:\arduino-maple\.
Open Programmers Notepad [WinAVR] from the start menu and then open the arduino-maple.cpp file from where it has been extracted. (C:\arduino-maple\arduino-maple.cpp in my case.)
If all is installed correctly, then you should be able to choose Tools - [WinAVR] Make All and have the following output in the output window:

> "make.exe" all
avr-gcc -Wall -g2 -gstabs -Os -fpack-struct -fshort-enums -ffunction-sections -fdata-sections -ffreestanding -funsigned-char -funsigned-bitfields -mmcu=atmega328p -DF_CPU=16000000UL -I./arduino -c arduino/pins_arduino.c -o build/arduino/pins_arduino.o
In file included from arduino/wiring_private.h:30,
                 from arduino/pins_arduino.c:26:
c:/winavr-20100110/lib/gcc/../../avr/include/avr/delay.h:36:2: warning: #warning "This file has been moved to <util/delay.h>."
...
Lots of warnings, no errors...
...
avr-g++ -Os -Wl,-gc-sections -mmcu=atmega328p  build/arduino/pins_arduino.o build/arduino/WInterrupts.o build/arduino/wiring.o build/arduino/wiring_analog.o build/arduino/wiring_digital.o build/arduino/wiring_pulse.o build/arduino/wiring_shift.o build/./arduino-maple.o build/arduino/HardwareSerial.o build/arduino/Print.o build/arduino/Tone.o build/arduino/WMath.o build/arduino/WString.o build/./libmaple.o -o app.elf
avr-objcopy -R .eeprom -O ihex app.elf  app.hex
avr-size --format=avr --mcu=atmega328p app.elf
AVR Memory Usage
----------------
Device: atmega328p

Program:    3380 bytes (10.3% Full)
(.text + .data + .bootloader)

Data:        796 bytes (38.9% Full)
(.data + .bss + .noinit)

> Process Exit Code: 0
> Time Taken: 00:02

Now, as you can see above... that's not my Arduino! I have an Arduino Mega and therefore we need to adjust the Makefile correctly for my setup. Here you'll need to know the CPU code, interface type and the port number. Also make sure you change the upload task to program so that we can use the menu items in WinAVR.

# Makefile for building small AVR executables, supports C and C++ code
# Author: Kiril Zyapkov
# Hacked up by nfd
 
SOURCE_DIRS = . arduino
INCLUDE_DIRS = arduino
MMCU = atmega1280
F_CPU = 16000000UL
SRC_ROOT = .
BUILD_DIR = build
 
CFLAGS = -Wall -g2 -gstabs -Os -fpack-struct -fshort-enums -ffunction-sections \
 -fdata-sections -ffreestanding -funsigned-char -funsigned-bitfields \
 -mmcu=$(MMCU) -DF_CPU=$(F_CPU) $(INCLUDE_DIRS:%=-I$(SRC_ROOT)/%)
 
CXXFLAGS = -Wall -g2 -gstabs -Os -fpack-struct -fshort-enums -ffunction-sections \
 -fdata-sections -ffreestanding -funsigned-char -funsigned-bitfields \
 -fno-exceptions -mmcu=$(MMCU) -DF_CPU=$(F_CPU) $(INCLUDE_DIRS:%=-I$(SRC_ROOT)/%)
 
LDFLAGS = -Os -Wl,-gc-sections -mmcu=$(MMCU) #-Wl,--relax
 
TARGET = $(notdir $(realpath $(SRC_ROOT)))
CC = avr-gcc
CXX = avr-g++
OBJCOPY = avr-objcopy
OBJDUMP = avr-objdump
AR  = avr-ar
SIZE = avr-size

SRC = $(wildcard $(SOURCE_DIRS:%=$(SRC_ROOT)/%/*.c))
 
CXXSRC = $(wildcard $(SOURCE_DIRS:%=$(SRC_ROOT)/%/*.cpp))

ASMSRC = $(wildcard $(SOURCE_DIRS:%=$(SRC_ROOT)/%/*.S))
 
OBJ = $(SRC:$(SRC_ROOT)/%.c=$(BUILD_DIR)/%.o) $(CXXSRC:$(SRC_ROOT)/%.cpp=$(BUILD_DIR)/%.o) $(ASMSRC:$(SRC_ROOT)/%.S=$(BUILD_DIR)/%.o)
 
DEPS = $(OBJ:%.o=%.d)
 
$(BUILD_DIR)/%.o: $(SRC_ROOT)/%.c
	$(CC) $(CFLAGS) -c $< -o $@

$(BUILD_DIR)/%.o: $(SRC_ROOT)/%.S
	$(CC) $(CFLAGS) -c $< -o $@
 
$(BUILD_DIR)/%.o: $(SRC_ROOT)/%.cpp
	$(CXX) $(CXXFLAGS) -c $< -o $@
 
all: app.hex printsize

#$(TARGET).a: $(OBJ)
#	$(AR) rcs $(TARGET).a $?

app.elf: $(OBJ)
	$(CXX) $(LDFLAGS) $(OBJ) -o $@
 
$(BUILD_DIR)/%.d: $(SRC_ROOT)/%.c
	mkdir -p $(dir $@)
	$(CC) $(CFLAGS) -MM -MF $@ $<
 
$(BUILD_DIR)/%.d: $(SRC_ROOT)/%.cpp
	mkdir -p $(dir $@)
	$(CXX) $(CXXFLAGS) -MM -MF $@ $<
 
#$(TARGET).elf: $(TARGET).a
#	$(CXX) $(LDFLAGS) $< -o $@
 
app.hex: app.elf
	$(OBJCOPY) -R .eeprom -O ihex $<  $@
 
 
clean:
	echo $(RM) $(DEPS) $(OBJ) $(TARGET).*
 
printsize:
	avr-size --format=avr --mcu=$(MMCU) app.elf

# Programming support using avrdude. Settings and variables.
PORT = /dev/tty.usbserial-A700ekGi
#PORT = /dev/ttyUSB0
AVRDUDE_PORT = com3
AVRDUDE_WRITE_FLASH = -U flash:w:app.hex
MCU = atmega1280
AVRDUDE_PROGRAMMER = stk500v1
#AVRDUDE_FLAGS = -V -F -C \app\arduino-0021\hardware\tools\avr\etc\avrdude.conf 
AVRDUDE_FLAGS = -V -F \
-p $(MCU) -P $(AVRDUDE_PORT) -c $(AVRDUDE_PROGRAMMER) \
-b $(UPLOAD_RATE)
UPLOAD_RATE = 57600
#
# Program the device.
INSTALL_DIR = \app\arduino-0021
AVRDUDE = avrdude
program: app.hex
	#python pulsedtr.py $(PORT)
	$(AVRDUDE) $(AVRDUDE_FLAGS) $(AVRDUDE_WRITE_FLASH)

The first line in the program target (Line 93) 'pushes' the reset button on the Arduino. As we don't have Python installed we have to do this manually. So, build the source code via the [WinAVR] Make All and ensure there are no errors. Now press the 'reset' button on the Arduino and then quickly choose Tools - [WinAVR] Program. If all goes correctly then you'll see the code uploading to your Arduino:

#python pulsedtr.py /dev/tty.usbserial-A700ekGi
avrdude -V -F -p atmega1280 -P com3 -c stk500v1 -b 57600 -U flash:w:app.hex
avrdude: AVR device initialized and ready to accept instructions

Reading | ################################################## | 100% 0.05s

avrdude: Device signature = 0x1e9703
avrdude: NOTE: FLASH memory has been specified, an erase cycle will be performed
         To disable this feature, specify the -D option.
avrdude: erasing chip
avrdude: reading input file "app.hex"
avrdude: input file app.hex auto detected as Intel Hex
avrdude: writing flash (3508 bytes):

Writing | ################################################## | 100% 1.09s

avrdude: 3508 bytes of flash written
avrdude done.  Thank you.

> Process Exit Code: 0
> Time Taken: 00:03

And that's it, our the compiled code for the arduino-maple project is now on the Arduino and running!

Using the Python code on Windows

I lied, I said I didn't have Python installed... but I ended up installing it anyway. The job to convert the Python code to C# was going to take too long and, to prevent a debugging nightmare, I decided I would get the known-to-work Python code going under Windows first. So, I installed Python from the official website and then started learning :)

Note that I downloaded and installed Python Version 2.6.6 as this was the version that would have been available when arduino-maple was created.

After selecting 'IDLE' from the start menu, I was presented with a light-weight GUI. I opened up the 'maple.py' that's included with the arduino-maple code and attempted to compile it.

I had problems at the start, as in the included Makefile there are two declarations of CPU type. Initially I hadn't set these both correctly and the Arduino includes were not correctly compiled... make sure you carry out a proper clean whenever changing code; this includes manually deleting all of the .o files.

Once this was sorted, it all finally worked... sort of?

connecting to COM3:
connected
SENT: 0000200121
No device found at address:
0x20
SENT: 0000010100
1c20000501000000ff0f3f000000000000000000415400ff204f544920313030746e6f436c6c6f7
2202072652020202020200024191bdc94191958dd481e508815481c9bdc99591b98da5308195
cdb995bdc91881154c8c9b40481051d491551394d25494130b14d1480b911508080808007d00371 117
Command 5 sender 0 recipient 20 length 1c
Device information:
Functions  : CONTROLLER
Periph 1   : 0x3f0fff
Periph 2   : 0x0
Periph 3   : 0x0
Name       : ..TAITO 001 Controller      $.  
License    : .....X...P.H..H..Y...S....\....[..T.....Q.I.I%M9M.0A.......P
Power      : 53251
Power max  : 32775
Play with the controller. Hit ctrl-c when done.
SENT: 010020090100000029
SENT: 010020090100000029
1111101010011111
SENT: 010020090100000029
1111101010011111
SENT: 010020090100000029
1111101010011111
SENT: 010020090100000029
1111101010011111

Timing issues

As I continued to run the code, I would get random results as to the 'Name', 'License' and 'Power' fields... well, it was obvious in the text fields that there were problems, but I had no idea that what the 'Power' fields were meant to read. Either way, this indicated a timing issue somewhere and I guessed that life was about to get difficult. Note that I had always been running the controller at 5v, as that's what WzDD's post said the blue wire should be connected to, but it turns out that the device should have actually been running on 3.3v.

Unfortunately, setting the controller on to 3.3v didn't change anything... the responses were still mildly random. The buttons would be sent though correctly (apart from the B button) but the initial device description would come through jumbled. I took a few samples of the data and realised that the data would always have a minimal length. To me this meant that we weren't missing bytes/bits, but actually reading too many. The analysis showed that there were similar chunks all the way through, but at certain intervals there'd be extra/changed characters. Based on Marcus Comstedt's Maple Bus information, I guessed that we were re-reading bits too quickly and needed to slow down a little... This would mean slowing down the pin reading in WzDD's arduino-maple assembler code.

In the maple_rx_raw: function down near _rx_loop:, the code iterates through the pins hoping to read the data. It initially checks the state of the pins to sense when the controller is about to send data, but this did not have the final check for the second pin going low.

4:	IN rport2, _SFR_IO_ADDR(PINB)
	BST rport2, bitmaple5			; maple5
	BRTS 4b							; must be low now

The above code was added around line 305 to ensure this check was in. I then also 'spaced' out the store/read calls by placing a delay in. WzDD had already written the delayquarter macro and I simply re-used this.

	...
	_rx_store rport bitmaple5 5
	_rx_read bitmaple5
	delayquarter
	_rx_store rport bitmaple1 4
	_rx_read bitmaple1
	...

After these two changes, the text from the device information call worked flawlessly. But my B button still didn't work. I decided the B button wasn't important at this stage and went on to decoding the controller to work out how the levers functioned.

Densha-De-Go controller workings

So, after having used the standard Dreamcast controller to play hours-upon-hours of Shenmue, Shenmue II and Chu chu rocket, I would have expected at least one of the levers on the Densha-De-Go controller to use the analog joystick and the other to also use some form of analog control. After a few minutes decoding the controls, it became apparent that they simply used the buttons available (up, down, left, right, x, y, z) in a binary-value style configuration.

Throttle Position X Y Z
T0 [x] [x]
T1 [x] [x]
T2 [x]
T3 [x] [x]
T4 [x]
T5 [x]
Other Buttons Mapping
SELECT D
START START
A A
B ??
C C
Brake Position UP DOWN LEFT RIGHT
B0 [x] [x] [x]
B1 [x] [x] [x]
B2 [x] [x]
B3 [x] [x] [x]
B4 [x] [x]
B5 [x] [x]
B6 [x]
B7 [x] [x] [x]
B8 [x] [x]
EM0 [x] [x]
EM1 [x]
EM2 [x] [x]
EM3 [x]
EM4 [x]
EM5 - - - -

In between the B's there is a slight overlap of the buttons, where the current and next 'combination' is combined, but this doesn't seem to last more than one cycle.
In between the EM's the controller reads UP-DOWN-LEFT-RIGHT. This area exists between all EM's and B8-EM0 and seems to be about 3mm wide.

Controlling trains

Now it was time to get this incorporated into my original train throttle. I didn't want to have to use the Python code, so instead I 'crafted' the packets into the Arduino code.

	//here is a quick packet to request device information from controller 'one'.
	//i.e. the only controller connected.
	packet.header[0] = 0; // Number of additional words in frame
	packet.header[1] = 0; // Sender address = Dreamcast
	packet.header[2] = (1 << 5); //(1 << 5); // Recipient address = main peripheral on port 0
	packet.header[3] = 1; // Command = request device information
	packet.data[0]   = 0x21; //checksum
	packet.data_len  = 5;
	//send the above packet to initiate comms with the controller.
	maple_transact();

I'm going to gloss over this part pretty quickly, as this post was more about getting the Dreamcast controller usable rather than another lesson in physics and model railway throttles. For the code, I ended up just deciphering the packet that comes back from the controller, working out what the throttle position is and then adjusting the target speed. The main program loop then makes then increment/decrements the current speed to eventually match the target speed. This provides a very simple form of acceleration.

void control_throttle(void) {
	//quick hack, should actually do a binary 'and'
	//packet data[6] contains the throttle position, so use this to gauge
	//voltage output to pin 7 (or speed)
	switch (packet.data[6]) {
		case 0xF9: target_speed = 0;	t_pos = 0; break;
		case 0xFA: target_speed = 88;	t_pos = 1; break;
		case 0xFB: target_speed = 96;	t_pos = 2; break;
		case 0xFC: target_speed = 140;	t_pos = 3; break;
		case 0xFD: target_speed = 190;	t_pos = 4; break;
		case 0xFE: target_speed = 250;	t_pos = 5; break;
	}
}

The changing of pin 13 below lights the on-board Arduino LED to show once the loop has made the speed match the target speed.

	digitalWrite(13, LOW);
	if (target_speed > speed) {
		//accel
		speed += t_pos / 2;
	} else if (target_speed < speed) {
		//speed += abs(target_speed - speed);
		//decel
		speed--;
	} else {
		digitalWrite(13, HIGH);
	}
	//send the speed to the pin/railway
	analogWrite(7, speed);

And then... it just worked :)

DSC03744

Download the source code here.
Note that this does not include the Python code; nor will my code correctly interact with the python code anymore. The goal was always to have the Arduino talk to the Dreamcast controller directly.

As you can see, I've only implemented the throttle on the controller... I need to now bring in the brake lever as well, but this is where the physics lesson comes in. I'll be posting again soon enough with source code for acceleration and braking using this controller.

I've got a few ideas as to how to semi-realistically control the train with both levers, but I will need to do a little more reading before I get something going. I do know that in the actual Densha-De-Go game, the game complains when you have both levers on at the same time... and it seems wrong to be doing so... but then again, it'd be like doing a hill-start :) Maybe for freight trains?

If only our model railway trains had gears and could free roll!

15Oct/102

Sending full bytes over .NET Serial Ports to the Arduino

Ok, I have just spent a good two nights of my life diagnosing why I could send all bytes up to value '127' and no higher. In hindsight, it all makes perfect sense and the documentation is clear, but when you've been taught to think in strings, you might hit this very quickly.

The Scenario

I have my MAX7219 + LedControl Library set up on my Arduino and all works fine. I use two functions to control it: setLed and setRow. setLed simply takes a boolean to determine if the LED you are pointing at is to be on or off, but setRow requires a byte. This is all fairly straight-forward as each 'row' in the LED matrix has 8 LEDs, and a byte has 8 bits. So, starting from the lowest significant bit, a value of b00000001 will turn on the first LED in a specified row. (i.e. setRow(DEVICE,ROW,BITS);).

All communications between my application and the Arduino had been based on strings and so I had previously been using one character (one byte) to set one LED. Due to this being a complete waste of bandwidth, I decided that each byte I sent through the channel should be a byte to control full row of LEDs. This meant that I could therefore no longer 'see' the output as a string (or ASCII), as the characters I would create from setting the bits may no longer be in ASCII range... this was no big deal, as I could just view the byte values and decode it all myself.

So, on the client end (C#.NET Application) I started encoding the bytes from bit values. This all worked until I tried to set the last bit...

byte b = 1 | (1 << 7); //let's set the first and last LED.
string buffer = (char)b + "\0";
serialPort.WriteLine(buffer);
Data Sent LEDs lit Correct?
b00000001 1st OK
b01010101 1st, 3rd, 5th, 7th OK
b10101010 1st, 2nd, 3rd, 4th, 5th, 6th WRONG
b10000001 1st, 2nd, 3rd, 4th, 5th, 6th WRONG
b01000000 7th OK
b10000000 1st, 2nd, 3rd, 4th, 5th, 6th WRONG

What the hell was going on? That 8th bit is fishy!

The Answer

So, after reading numerous blogs and not finding my answer, I went to the Arduino Forums and posted a topic asking for help. I was given advice to write a very simple test app to work out where the bytes were failing... but I never did get to write that app, instead I went to the MSDN site as soon as I saw that the Write() procedure could be overloaded.

And look what I found at the article on MSDN:

By default, SerialPort uses ASCIIEncoding to encode the characters. ASCIIEncoding encodes all characters greater than 127 as (char)63 or '?'. To support additional characters in that range, set Encoding to UTF8Encoding, UTF32Encoding, or UnicodeEncoding.

And guess what... ASCII Character ? is 63 in decimal and therefore b00111111 in binary!
So, whenever I was setting the 8th bit, the .NET Framework (in all its wisdom) would translate this to a question mark as it was not expecting to send an invalid ASCII character. Ladies and Gentlemen, ASCII is only 7 bits!

The work-around?

byte[] b = new byte[] { 1, 127, 128, 129, 255 }; //let's set the first bit, last bit, etc...
serialPort.Write(b, 0, b.Length);

And then everything just worked. Do not send chars to your port if your receiver wants bytes.

17Aug/102

Adding LEDs to a Japanese Shop

Pictures speak louder than words, so below is a quick tutorial on how to get an off-the-shelf TomyTec Japanese Shop Building lit with LED lighting. In total, this building received 6 LEDs; lantern, side-door, top floor (x2), bottom floor (x2).

Interior Lighting + Side Door

Model Railway Model Railway Model Railway

The trickiest part of this installation was the lantern that hangs out the front. I actually sliced it in half and bored out the middle to fit an LED inside. I also trimmed down the LED with a file to get it to fit a little more easily. This was done with my pocket-knife and I stopped when I felt it grinding metal. :)

Note that I borrowed ideas from this blog and I strongly recommend you check out the work the author has done on their layout!

Front Lantern

Model Railway Model Railway Model Railway
Model Railway Model Railway

And finally, everything is wired up. You can see the huge hole I accidently drilled in the side of the shop... luckily the lantern covers it over pretty well.

Finished Product

Model Railway Model Railway Model Railway