modelrail.otenko Trains, computers, vintage, retro, model railways… the lot…

Smushing JSON into submission!

I don't even know how to describe this operation on JSON. The basic idea is that we want to deserialise a stream into something legible, without creating a class for each item in an array. json2csharp believes that each item should be an array since it sees them all as individually named classes. Let me prove it! Here's a chunk'o'data:

{
    "trainCount":122,
    "requestTimestamp":1601124556789,
    "responseTimestamp":1601128156789,
    "nextUrl":"https://junatkartalla-cal-prod.herokuapp.com/trains/1601128156789",
    "trains":{
        "8":{
			"id":"8",
			"from":"LR",
			"to":"HKI",
			"title":"IC8",
			"latitude":60.172097,
			"longitude":24.941249,
			"speed":0,
			"direction":0,
			"category":"IC",
			"status":"1",
			"delta":60,
			"trainType":"LONGDISTANCE",
			"updated":1601127783841,
			"action":"deleted"
        },
        "9":{
			"id":"9",
			"from":"HKI",
			"to":"LR",
			"title":"S9",
			"latitude":60.571148,
			"longitude":25.199523,
			"speed":0,
			"direction":0,
			"category":"S",
			"status":"1",
			"delta":60,
			"trainType":"LONGDISTANCE",
			"updated":1601128143878,
			"action":"modified"
        }
    }
}

So, if you slam that JSON into json2csharp, you'll get the following:

    public class 8    {
        public string id { get; set; } 
        public string from { get; set; } 
        public string to { get; set; } 
        public string title { get; set; } 
        public double latitude { get; set; } 
        public double longitude { get; set; } 
        public int speed { get; set; } 
        public int direction { get; set; } 
        public string category { get; set; } 
        public string status { get; set; } 
        public int delta { get; set; } 
        public string trainType { get; set; } 
        public long updated { get; set; } 
        public string action { get; set; } 
    }

    public class 9    {
        public string id { get; set; } 
        public string from { get; set; } 
        public string to { get; set; } 
        public string title { get; set; } 
        public double latitude { get; set; } 
        public double longitude { get; set; } 
        public int speed { get; set; } 
        public int direction { get; set; } 
        public string category { get; set; } 
        public string status { get; set; } 
        public int delta { get; set; } 
        public string trainType { get; set; } 
        public long updated { get; set; } 
        public string action { get; set; } 
    }

    public class Trains    {
        public 8 8 { get; set; } 
        public 9 9 { get; set; } 
    }

    public class Root    {
        public int trainCount { get; set; } 
        public long requestTimestamp { get; set; } 
        public long responseTimestamp { get; set; } 
        public string nextUrl { get; set; } 
        public Trains trains { get; set; } 
    }

So, that's actually uncompilable code. Is uncompilable a word? Dunno... this is actually the first time that json2csharp has failed me! No matter the options selected on the site, the output was always bad code... json2csharp doesn't work with ALL json! So, what to do? Well, we actually need to mangle this JSON into submission. The best bet would be to move that train id/number into the array object as a parameter, rather than having it as the dictionary key. We have two methods to do this...

Using jQuery MAP Function

If you are pulling JSON from a hosted browser, then you can run JS in the browsers console and produce cleaner JSON. In this case, you want to use jQuery's MAP function to rewrite each array object:

json_data.trains = $.map(json_data.trains, function (data, idx) { return data; });

If you feed the JSON at the top of this post into that function, you'll get the following:

{
	"trainCount": 122,
	"requestTimestamp": 1601124556789,
	"responseTimestamp": 1601128156789,
	"nextUrl": "https://junatkartalla-cal-prod.herokuapp.com/trains/1601128156789",
	"trains": [{
		"id": "8",
		"from": "LR",
		"to": "HKI",
		"title": "IC8",
		"latitude": 60.172097,
		"longitude": 24.941249,
		"speed": 0,
		"direction": 0,
		"category": "IC",
		"status": "1",
		"delta": 60,
		"trainType": "LONGDISTANCE",
		"updated": 1601127783841,
		"action": "deleted"
	}, {
		"id": "9",
		"from": "HKI",
		"to": "LR",
		"title": "S9",
		"latitude": 60.571148,
		"longitude": 25.199523,
		"speed": 0,
		"direction": 0,
		"category": "S",
		"status": "1",
		"delta": 60,
		"trainType": "LONGDISTANCE",
		"updated": 1601128143878,
		"action": "modified"
	}]
}

Very nice.

Using C# Regex

I've never thoroughly learnt Regex and it still makes me sad. One day I'll sit down and go through a course. For now, googlin' works nicely to find an example for this scenario. Effectively we want to remove the "NUM": and just leave the curly braces. We then also need to remove a curly brace at the end.

            var data = (new System.Net.WebClient()).DownloadString(nextUrl);
            data = new Regex("\"([0-9]|[1-9][0-9]|[1-9][0-9][0-9]|[1-9][0-9][0-9][0-9]|[1-9][0-9][0-9][0-9][0-9])\":").Replace(data, "");
            data = data.Replace("\"trains\":{", "\"trains\":[");
            data = data.Replace("}}}", "}]}");

So, the first line downloads the data. The second uses Regex to find anything in double-quotes that's a number from 0-9. I actually wanted 0-9999, but it seems you need to match each character 1-by-1, so I gave it options with | (pipe) up to 99999. From there, the third line turns the trains object into an array and the final line closes that array with a square bracket instead of the existing curly brace.

Final C# Class Output

With either of the methods above, you end up with cleaned JSON. From here, slapping that back in json2csharp gets you the following:

    public class Train    {
        public string id { get; set; } 
        public string from { get; set; } 
        public string to { get; set; } 
        public string title { get; set; } 
        public double latitude { get; set; } 
        public double longitude { get; set; } 
        public int speed { get; set; } 
        public int direction { get; set; } 
        public string category { get; set; } 
        public string status { get; set; } 
        public int delta { get; set; } 
        public string trainType { get; set; } 
        public object updated { get; set; } 
        public string action { get; set; } 
    }

    public class Root    {
        public int trainCount { get; set; } 
        public long requestTimestamp { get; set; } 
        public long responseTimestamp { get; set; } 
        public string nextUrl { get; set; } 
        public List<Train> trains { get; set; } 
    }

Thank you json2charp, that's exactly the reusability that I was looking for!