modelrail.otenko Trains, computers, vintage, retro, model railways… the lot…

Streaming from a Sony HDR-AS100V Action Cam

This is a cool camera. I picked it up from the Hard Off in Kagoshima whilst I was visiting the southernmost JR Railway station in Japan. I then used it to take a miriad of videos, especially from the balcony at the apartment in Shin Osaka.

Whilst it works great recording by itself, you can also use the Play Mobile (now Imaging Edge Mobile) application from Sony to control it remotely. The application uses the phones Wifi to connect to the camera, which acts as a wifi access point. From there, it shows a live view of what the camera can see whilst you're trying to line up the next shot. You can even control the camera remotely with the buttons on screen, being able to start and stop recording very easily... especially without bumping the camera!

I was hoping Sony would release the same software for the PC and allow me to connect wirelessly to the camera. This would then let me sniff the traffic and get the video stream live link, but alas, it's only available for mobile devices. Wait!, you say, there is a version of Play Memories for windows! Sure, go ahead and download it... install it even! It'll connect to your camera, but there's no ability to see the live stream from the lens... I tried... trust me....

With the USB camera connected, you'll get the screens above. They let you update firmware and manage settings, but not much else. Of course, there's also the file transfer functionality to let you download the images and videos you've taken. Interestingly, the last image shows the second pane of the Network Settings tab which allows you to configure the camera to upload to UStream automatically. Turns out that this was a 'thing' back in the day and not really practical anymore: IBM bought UStream and the service is no longer free... nor could I get it to work, even with a trial account.

Can we create a network to inspect the Wifi?

So, I had a plan. I have a Vonets VAR11N-300 that allows an ethernet cabled device to access an access point over Wifi. With this, I could use an ethernet port on my laptop to connect to the camera. The Wifi adapter in the laptop could then create an ad-hoc network that my phone could connect to. If I bridged the ethernet to the Wifi, connected my phone ad-hoc to the wifi network created by the laptop, then I could possibly pass the traffic through the laptop and use wireshark to listen in?

As I was plugging this all together, traffic started flowing without client application requesting anything! I'd not yet bridged the adapters, but I'd installed Wireshark and had the VAR11N-300 connected to the camera and, all of a sudden, I'd noticed jitters on the traffic graph for the ethernet interface on the front page of Wireshark. The ethernet port seems to have received an IP from the camera since it was acting as a terminating host and not an ethernet bridge! For everyones information: the IP range is 196.168.0.x, with IPs dished out from 1 and the camera being 192.168.122.1. With the little graph jumping around, I decided to listen in and have a look!

Ok so, from left to right above... the Wireshark main screen with my miriad of interfaces. You may notice that I have Wifi-2 selected there and there's traffic flowing on it... At this point, realising that I didn't need the mobile application to trigger data flow from the camera, I'd slapped a USB Wifi dongle into my main desktop so that I could work in comfort. The second screen is the main window of Wireshark once you're sniffing. You'll see a huge list of traffic up the top. Wireshark, without filters, will just present everything it can possibly see. Each row is a packet sent over the Wifi and you may need to combine multiple rows to see an entire picture, depending on what you're looking for. Usually, if this was my main network interface, I'd use a filter at the top with either ip.dest == 192.x.x.x or ip.src == 192.x.x.x to limit the displayed data, but fortunately this interface was only talking to the camera.

The second shot there has a very interesting packet displayed. It turns out that the camera is broadcasting UPnP information, telling anyone listening on the immediate network how to find it! No wonder, once you've connected your phone to the camera, that it can find it. It doesn't even have to look as the camera just yells it out every few seconds. Let's check out the data we're receiving...

<?xml version="1.0" encoding="utf-8"?>
<root xmlns="urn:schemas-upnp-org:device-1-0" xmlns:dlna="urn:schemas-dlna-org:device-1-0" xmlns:av="urn:schemas-sony-com:av">
	<specVersion>
		<major>1</major>
		<minor>0</minor>
	</specVersion>
	<device>
		<dlna:X_DLNADOC xmlns:dlna="urn:schemas-dlna-org:device-1-0">DMS-1.50</dlna:X_DLNADOC>
		<deviceType>urn:schemas-upnp-org:device:MediaServer:1</deviceType>
		<friendlyName>HDR-AS100V</friendlyName>
		<manufacturer>Sony Corporation</manufacturer>
		<manufacturerURL>http://www.sony.com/</manufacturerURL>
		<modelDescription>SonyDigitalMediaServer</modelDescription>
		<modelName>SonyImagingDevice</modelName>
		<modelURL>http://www.sony.net/</modelURL>
		<UDN>uuid:00000000-0005-0010-8000-fcc2de658d5e</UDN>
		<serviceList>
			<service>
				<serviceType>urn:schemas-upnp-org:service:ContentDirectory:1</serviceType>
				<serviceId>urn:upnp-org:serviceId:ContentDirectory</serviceId>
				<SCPDURL>/CdsDesc.xml</SCPDURL>
				<controlURL>/upnp/control/ContentDirectory</controlURL>
				<eventSubURL>/upnp/event/ContentDirectory</eventSubURL>
			</service>
			<service>
				<serviceType>urn:schemas-upnp-org:service:ConnectionManager:1</serviceType>
				<serviceId>urn:upnp-org:serviceId:ConnectionManager</serviceId>
				<SCPDURL>/CmsDesc.xml</SCPDURL>
				<controlURL>/upnp/control/ConnectionManager</controlURL>
				<eventSubURL>/upnp/event/ConnectionManager</eventSubURL>
			</service>
			<service>
				<serviceType>urn:schemas-sony-com:service:ScalarWebAPI:1</serviceType>
				<serviceId>urn:schemas-sony-com:serviceId:ScalarWebAPI</serviceId>
				<SCPDURL>/ScalarWebApiDesc.xml</SCPDURL>
				<controlURL/>
				<eventSubURL/>
			</service>
		</serviceList>
		<iconList>
			<icon>
				<mimetype>image/jpeg</mimetype>
				<width>48</width>
				<height>48</height>
				<depth>24</depth>
				<url>/DLNA_camera_48.jpg</url>
			</icon>
			<icon>
				<mimetype>image/jpeg</mimetype>
				<width>120</width>
				<height>120</height>
				<depth>24</depth>
				<url>/DLNA_camera_120.jpg</url>
			</icon>
			<icon>
				<mimetype>image/png</mimetype>
				<width>48</width>
				<height>48</height>
				<depth>24</depth>
				<url>/DLNA_camera_48.png</url>
			</icon>
			<icon>
				<mimetype>image/png</mimetype>
				<width>120</width>
				<height>120</height>
				<depth>24</depth>
				<url>/DLNA_camera_120.png</url>
			</icon>
		</iconList>
		<av:X_ScalarWebAPI_DeviceInfo xmlns:av="urn:schemas-sony-com:av">
			<av:X_ScalarWebAPI_Version>1.0</av:X_ScalarWebAPI_Version>
			<av:X_ScalarWebAPI_ServiceList>
				<av:X_ScalarWebAPI_Service>
					<av:X_ScalarWebAPI_ServiceType>guide</av:X_ScalarWebAPI_ServiceType>
					<av:X_ScalarWebAPI_ActionList_URL>http://192.168.122.1:10000/sony</av:X_ScalarWebAPI_ActionList_URL>
					<av:X_ScalarWebAPI_AccessType />
				</av:X_ScalarWebAPI_Service>
				<av:X_ScalarWebAPI_Service>
					<av:X_ScalarWebAPI_ServiceType>accessControl</av:X_ScalarWebAPI_ServiceType>
					<av:X_ScalarWebAPI_ActionList_URL>http://192.168.122.1:10000/sony</av:X_ScalarWebAPI_ActionList_URL>
					<av:X_ScalarWebAPI_AccessType />
				</av:X_ScalarWebAPI_Service>
				<av:X_ScalarWebAPI_Service>
					<av:X_ScalarWebAPI_ServiceType>camera</av:X_ScalarWebAPI_ServiceType>
					<av:X_ScalarWebAPI_ActionList_URL>http://192.168.122.1:10000/sony</av:X_ScalarWebAPI_ActionList_URL>
					<av:X_ScalarWebAPI_AccessType />
				</av:X_ScalarWebAPI_Service>
			</av:X_ScalarWebAPI_ServiceList>
			<av:X_ScalarWebAPI_ImagingDevice>
				<av:X_ScalarWebAPI_LiveView_URL>http://192.168.122.1:60152/liveview.JPG?%211234%21http%2dget%3a%2a%3aimage%2fjpeg%3a%2a%21%21%21%21%21</av:X_ScalarWebAPI_LiveView_URL>
				<av:X_ScalarWebAPI_DefaultFunction>RemoteShooting</av:X_ScalarWebAPI_DefaultFunction>
			</av:X_ScalarWebAPI_ImagingDevice>
		</av:X_ScalarWebAPI_DeviceInfo>
	</device>
</root>

Oooowwweee... Check that link at the bottom!? X_ScalarWebAPI_LiveView_URL? Really? Can it be this easy? I slapped it into a browser and no... it's not this easy... the browser just doesn't respond as it's constantly being fed a stream of data and doesn't know what to do with it! Turns out this stream is a motion-jpeg or some frame-by-frame binary flow that isn't a standard video stream. So, what to do? Google the hell out of those keywords and see who has already come to our rescue!

First hit was a post on stackoverflow asking 'How to get Sony ScalarWebAPI method list'. From the responses, one person suggested to send the camera JSON via a post so that you can see what functions you can call. One such function is getMethodTypes and can be called via the following post body:

{"method": "getMethodTypes", "params": [""], "id": 1, "version": "1.0"}

Using Postman, I got the following result...

{
    "id": 1,
    "results": [
        [
            "actTakePicture",
            [],
            [
                "string*"
            ],
            "1.0"
        ],
        [
            "getApplicationInfo",
            [],
            [
                "string",
                "string"
            ],
            "1.0"
        ],
        [
            "getAvailableApiList",
            [],
            [
                "string*"
            ],
            "1.0"
        ],
        [
            "getAvailableCameraFunction",
            [],
            [
                "string",
                "string*"
            ],
            "1.0"
        ],
        [
            "getAvailableMovieQuality",
            [],
            [
                "string",
                "string*"
            ],
            "1.0"
        ],
        [
            "getAvailableShootMode",
            [],
            [
                "string",
                "string*"
            ],
            "1.0"
        ],
        [
            "getAvailableSteadyMode",
            [],
            [
                "string",
                "string*"
            ],
            "1.0"
        ],
        [
            "getCameraFunction",
            [],
            [
                "string"
            ],
            "1.0"
        ],
        [
            "getEvent",
            [
                "bool"
            ],
            [
                "{\"type\":\"string\", \"names\":\"string*\"}",
                "{\"type\":\"string\", \"cameraStatus\":\"string\"}",
                "{\"type\":\"string\", \"zoomPosition\":\"int\", \"zoomNumberBox\":\"int\", \"zoomIndexCurrentBox\":\"int\", \"zoomPositionCurrentBox\":\"int\"}",
                "{\"type\":\"string\", \"liveviewStatus\":\"bool\"}",
                "{\"type\":\"string\", \"liveviewOrientation\":\"string\"}",
                "{\"type\":\"string\", \"takePictureUrl\":\"string*\"}*",
                "{\"type\":\"string\", \"continuousError\":\"string\", \"isContinued\":\"bool\"}*",
                "{\"type\":\"string\", \"triggeredError\":\"string*\"}",
                "{\"type\":\"string\", \"sceneRecognition\":\"string\", \"steadyRecognition\":\"string\", \"motionRecognition\":\"string\"}",
                "{\"type\":\"string\", \"formatResult\":\"string\"}",
                "{\"type\":\"string\", \"storageID\":\"string\", \"recordTarget\":\"bool\", \"numberOfRecordableImages\":\"int\", \"recordableTime\":\"int\", \"storageDescription\":\"string\"}*",
                "{\"type\":\"string\", \"currentBeepMode\":\"string\", \"beepModeCandidates\":\"string*\"}",
                "{\"type\":\"string\", \"currentCameraFunction\":\"string\", \"cameraFunctionCandidates\":\"string*\"}",
                "{\"type\":\"string\", \"currentMovieQuality\":\"string\", \"movieQualityCandidates\":\"string*\"}",
                "{\"type\":\"string\", \"checkAvailability\":\"bool\", \"currentAspect\":\"string\", \"currentSize\":\"string\"}",
                "{\"type\":\"string\", \"cameraFunctionResult\":\"string\"}",
                "{\"type\":\"string\", \"currentSteadyMode\":\"string\", \"steadyModeCandidates\":\"string*\"}",
                "{\"type\":\"string\", \"currentViewAngle\":\"int\", \"viewAngleCandidates\":\"int*\"}",
                "{\"type\":\"string\", \"currentExposureMode\":\"string\", \"exposureModeCandidates\":\"string*\"}",
                "{\"type\":\"string\", \"currentPostviewImageSize\":\"string\", \"postviewImageSizeCandidates\":\"string*\"}",
                "{\"type\":\"string\", \"currentSelfTimer\":\"int\", \"selfTimerCandidates\":\"int*\"}",
                "{\"type\":\"string\", \"currentShootMode\":\"string\", \"shootModeCandidates\":\"string*\"}",
                "{\"type\":\"string\", \"currentAELock\":\"bool\", \"aeLockCandidates\":\"bool*\"}",
                "{\"type\":\"string\", \"checkAvailability\":\"bool\", \"currentBracketShootMode\":\"string\", \"currentBracketShootModeOption\":\"string\"}",
                "{\"type\":\"string\", \"checkAvailability\":\"bool\", \"currentCreativeStyle\":\"string\", \"currentCreativeStyleContrast\":\"int\", \"currentCreativeStyleSaturation\":\"int\", \"currentCreativeStyleSharpness\":\"int\"}",
                "{\"type\":\"string\", \"currentExposureCompensation\":\"int\", \"maxExposureCompensation\":\"int\", \"minExposureCompensation\":\"int\", \"stepIndexOfExposureCompensation\":\"int\"}",
                "{\"type\":\"string\", \"currentFlashMode\":\"string\", \"flashModeCandidates\":\"string*\"}",
                "{\"type\":\"string\", \"currentFNumber\":\"string\", \"fNumberCandidates\":\"string*\"}",
                "{\"type\":\"string\", \"currentFocusMode\":\"string\", \"focusModeCandidates\":\"string*\"}",
                "{\"type\":\"string\", \"currentIsoSpeedRate\":\"string\", \"isoSpeedRateCandidates\":\"string*\"}",
                "{\"type\":\"string\", \"checkAvailability\":\"bool\", \"currentPictureEffect\":\"string\", \"currentPictureEffectOption\":\"string\"}",
                "{\"type\":\"string\", \"isShifted\":\"bool\"}",
                "{\"type\":\"string\", \"currentShutterSpeed\":\"string\", \"shutterSpeedCandidates\":\"string*\"}",
                "{\"type\":\"string\", \"checkAvailability\":\"bool\", \"currentWhiteBalanceMode\":\"string\", \"currentColorTemperature\":\"int\"}",
                "{\"type\":\"string\", \"currentSet\":\"bool\", \"currentTouchCoordinates\":\"double*\"}"
            ],
            "1.0"
        ],
        [
            "getMethodTypes",
            [
                "string"
            ],
            [
                "string",
                "string*",
                "string*",
                "string"
            ],
            "1.0"
        ],
        [
            "getMovieQuality",
            [],
            [
                "string"
            ],
            "1.0"
        ],
        [
            "getShootMode",
            [],
            [
                "string"
            ],
            "1.0"
        ],
        [
            "getSteadyMode",
            [],
            [
                "string"
            ],
            "1.0"
        ],
        [
            "getStorageInformation",
            [],
            [
                "{\"storageID\":\"string\", \"recordTarget\":\"bool\", \"numberOfRecordableImages\":\"int\", \"recordableTime\":\"int\", \"storageDescription\":\"string\"}*"
            ],
            "1.0"
        ],
        [
            "getSupportedCameraFunction",
            [],
            [
                "string*"
            ],
            "1.0"
        ],
        [
            "getSupportedMovieQuality",
            [],
            [
                "string*"
            ],
            "1.0"
        ],
        [
            "getSupportedShootMode",
            [],
            [
                "string*"
            ],
            "1.0"
        ],
        [
            "getSupportedSteadyMode",
            [],
            [
                "string*"
            ],
            "1.0"
        ],
        [
            "getVersions",
            [],
            [
                "string*"
            ],
            "1.0"
        ],
        [
            "setCameraFunction",
            [
                "string"
            ],
            [
                "int"
            ],
            "1.0"
        ],
        [
            "setMovieQuality",
            [
                "string"
            ],
            [
                "int"
            ],
            "1.0"
        ],
        [
            "setShootMode",
            [
                "string"
            ],
            [
                "int"
            ],
            "1.0"
        ],
        [
            "setSteadyMode",
            [
                "string"
            ],
            [
                "int"
            ],
            "1.0"
        ],
        [
            "startIntervalStillRec",
            [],
            [
                "int"
            ],
            "1.0"
        ],
        [
            "startLiveview",
            [],
            [
                "string"
            ],
            "1.0"
        ],
        [
            "startMovieRec",
            [],
            [
                "int"
            ],
            "1.0"
        ],
        [
            "stopIntervalStillRec",
            [],
            [
                "int"
            ],
            "1.0"
        ],
        [
            "stopLiveview",
            [],
            [
                "int"
            ],
            "1.0"
        ],
        [
            "stopMovieRec",
            [],
            [
                "string"
            ],
            "1.0"
        ]
    ]
}

So, there's a startLiveview command down the bottom there... maybe I need to call this prior to trying to actually display the URL in a browser? It easily be called via Postman with the body:

{"method": "startLiveview","params" : [],"id" : 1,"version" : "1.0"}

You'll get the following result...

{
    "id": 1,
    "result": [
        "http://192.168.122.1:60152/liveview.JPG?%211234%21http%2dget%3a%2a%3aimage%2fjpeg%3a%2a%21%21%21%21%21"
    ]
}

We knew this link already from the XML above, but does the video now work? I slapped the URL into the browser but didn't have much luck. So... a little more digging and I found the following: Tony Jan's work on decode the Sony stream written in Ruby, kazyx's kz-remote-api written in C# and twenzel's version called SonyCameraRemoteControl also in C#.Finally, with the term 'liveview' added to my search, I hit cryptofuture's liveView project which contained binaries to run really easy on Windows.

Horrible image quality... but we have it! A livestream from the HDR-AS100V! I wrapped it up at that point, since you can see it was past midnight.

Getting this out to the world!

Nice, we have a connection. Next step? Pipe this to Youtube. I was going to use a Raspberry Pi, so Ruby will an easy choice and therefore Tony's library was tested. Gem is installed by default...

pi@raspberrypi:~ $ sudo gem install sonycam
Fetching: thor-1.0.1.gem (100%)
Successfully installed thor-1.0.1
Fetching: sonycam-1.3.2.gem (100%)
Successfully installed sonycam-1.3.2
Parsing documentation for thor-1.0.1
Installing ri documentation for thor-1.0.1
Parsing documentation for sonycam-1.3.2
Installing ri documentation for sonycam-1.3.2
Done installing documentation for thor, sonycam after 3 seconds
2 gems installed

Just for fun, after hooking up the wireless... try to take a picture?

pi@raspberrypi:~ $ sonycam api actTakePicture
Can not find /home/pi/.sonycam, start scanning...
Found location: http://192.168.122.1:64321/DmsRmtDesc.xml
Device description file saved to /home/pi/.sonycam
Traceback (most recent call last):
        8: from /usr/local/bin/sonycam:23:in `<main>'
        7: from /usr/local/bin/sonycam:23:in `load'
        6: from /var/lib/gems/2.5.0/gems/sonycam-1.3.2/bin/sonycam:3:in `<top (required)>'
        5: from /var/lib/gems/2.5.0/gems/thor-1.0.1/lib/thor/base.rb:485:in `start'
        4: from /var/lib/gems/2.5.0/gems/thor-1.0.1/lib/thor.rb:392:in `dispatch'
        3: from /var/lib/gems/2.5.0/gems/thor-1.0.1/lib/thor/invocation.rb:127:in `invoke_command'
        2: from /var/lib/gems/2.5.0/gems/thor-1.0.1/lib/thor/command.rb:27:in `run'
        1: from /var/lib/gems/2.5.0/gems/sonycam-1.3.2/lib/sonycam/cli.rb:49:in `api'
/var/lib/gems/2.5.0/gems/sonycam-1.3.2/lib/sonycam/api.rb:17:in `request': Sonycam::Error::IllegalRequest

I suppose, this is an Action Cam and not an SLR? Let's try the livestream to MP4.

sonycam liveview | ffmpeg -f image2pipe -c mjpeg -i pipe:0 -codec copy liveview.mp4

Trying to open the video file whilst the recording was happening didn't work... but it opened successfully once it was stopped.

The final option was to use ffserver, but it turns out that it's ancient technology that's been deprecated in FFmpeg. I quickly looked at the age of the repository and noticed it was last updated 6 years ago!

In the end, it turns out that the liveview is only ever outputted at 635x300-something. It's not HD! Nooooo... alllll... thissss... timmmeeee... speennntt.... It seems that there's another call for startLiveviewWithSize, but my camera doesn't support this. Oh right, only the full-frame cameras do!? And look at that link, they've called Liveview "viewfinder"! Of course it wont be full resolution.

HDMI Output

What goes out, must come in? Sure, this thing has a mini/micro/something HDMI port in its ass... but how does that help me? YouTube doesn't have a HDMI input plug! Well, eBay does have crappy HDMI to USB converters.. so..

I bought that thing... and got this input via VLC. The actual software they wanted to use needed Direct X Movie/Video WMV software installed and Windows 10 didn't want to even care.

Image quality was OK, but no colour? I didn't bother digging further as I really didn't want HDMI cables running all over the shop.

Let's just use the old camera

Oh well.. it's back to the old camera until I find another method of getting this camera to stream. Here it is, right now, streaming live to the world from my balcony.

Hahahaa... do you like my handywork? (Go over here to Youtube if the embed below is broken.)

I'll update if I manage to find a better method. There's hints of people hacking apart the APK (yes, these cameras run Android!) on the unit that streams to UStream.
Why can't we just create one that provides an RTSP endpoint, like my camera above?