Effect of Frame & Bit rates on Latency

Extract

Back in March, an update to the Raspberry Pi camera software introduced some new video modes with higher frame rates for resolutions below full 1080p HD.  For 720p HD, frame rates up to 49 fps are now available and for VGA (640×480) there are new modes for 60 and 90 fps.  There were reports that, at these higher frame rates, the latency was reduced.  This didn’t come as a surprise as it has been known for a while that the GoPro video out has lower latency at 48 fps than at 30 fps.  The reasons for this are not totally clear, but is thought to be due to the differences between frames being less at the higher frame rates allowing for more efficient compression.

In order to investigate this I ran some back-to-back tests at various frame rates.  I was expecting that with twice as many frames being squeezed into the same bit rates, the video quality would suffer, so the tests where repeated at three different bit rates.  The result of this testing was a 25% reduction in latency, with a new minimum of 121 milliseconds at 720p and 48 fps.  As expected the video quality at the old bit rate of 2.5Mbps suffered for fast moving backgrounds, but even when increased to 6.0Mbps, the latency was still only 126 Mbps; a 23% reduction.

With VGA resolution at 60fps and 2.5 Mbps the latency dropped to 87 ms and while this is useful, in terms of hardware size, convenience and even latency, it is still bettered by dedicated FPV hardware.  If you include the cost of the laptop, it looses out on price too.  Where the Pi has the advantage is at the 1280 x 720p HD resolution where there is currently no affordable competition.

Setup

The test environment was set up the same as previously, with the camera pointing at the laptop screen.  The camera was positioned so that at least ten iterations of the image were visible at once. I used a single LED torch as the measurement indicator.  The tests were run with no overclocking.

I used a UDP stream with some improved scripts to simplify running all the variations. The scripts can be found on the Current Scripts Page.

A GoPro 3 Black was used to record the screen at 120 fps.

Tests were run at frame rates of 25, 30, 36, 42, 45, 48 and 49 fps and at bit rates of 2.5, 4.5 and 6.0 Mbps.   Up to 5 LED on/led off cycles were recorded on the GoPro for later analysis.  With 10 iteration for each of the 5 on/offs, the calculated latency represents the average of 100 tests.

Results 1 – Latency

Across all the bit rates, the results show a definite reduction in latency with increasing frame rate right up to 48 frames per second. At 49 fps the latency increased slightly. The lowest latency achieved was 121ms at 48 fps for the 2.5 Mbps stream.  A reduction of 25% over the 25 fps result.  The 6.0 Mps stream still managed a reduced latency of 126 ms at 48 fps and the 4.5 Mbps stream achieved 125 ms.

The VGA test showed a much lower latency at 87ms, even though it was using the same bit rate as the smallest 720p HD stream.

The graph shows some odd behavior between the 4.5 and 6.0 Mbps curves.  Below about 40 fps the 4.5 Mbps stream has higher latency than the 6.0 Mbps stream.  Beyond 40 fps the curves switch over to what you would expect.

fps

In addition to the average latency, the graph below shows the minimum and maximums for each bit rate.  What is apparent is that there is a lot of overlap between the results and that the latency can vary +/- 20 fps, particularly at the lower bit rates.

fps2

Results 2 – Video Quality

The frame grabs below show a clear difference between the three bit rates at 48 fps. The 2.5 Mbps stream shows extreme pixelation and a general blurring of the image.

Quality25

Frame grab from 2.5 Mbps stream @ 48 fps

The  4.5 Mbps shows a reduction in the pixelation.  This is apparent in the sky and on the tiled roof.

Quality45

Frame grab from 4.5 Mbps Stream @ 48 fps

The 6.0 Mbps is the clear winner, with a minimum of pixelation and the sharpest image.

Quality60

Frame grab from 6.0 Mbps Stream @ 48 fps

Conclusion

The total latency is made up of capture, compression, transmission and display components.  The capture and  display components should be the same for the three bit rates as the resolution is the same.  The transmission component should increase in relation to the higher bit rates.  That just leaves the compression component.  The compressor has to work hard to squeeze the stream into a smaller bit rate whilst maintaining the best video quality possible.  For this reason decreasing the bit rate can increase the latency, although there is probably a point where the extreme pixelation starts reducing the latency again.

These opposing affects between compression and transmission are likely what has caused the odd behavior between the 4.5 Mbps and 6.0 Mbps streams.  At the lower frame rates, the compression delay for the 4.5 Mbps stream was more significant.  At the higher frame rates the transmission delay in the 6.0 Mbps stream became more significant.

While the VGA test used the same 2.5 Mbps bit rate as the HD test, with only a third of the pixels, the capture, compression and display components resulted in a much lower latency of 87 ms.

While the lowest latency was achieved for the 2.5 Mbps stream, the extreme pixelation of the image renders it pretty much unusable.  The 4.5 Mbps stream could be used but it only has a 1 ms advantage over the superior image quality of the 6.0 Mbps stream.  For this reason I plan to adopt the 48 fps, 6.0 Mbps, 126 ms stream as my new baseline when flying.

I will probably also switch to a 24 fps pipeline for youtube videos to get the best image quality from the 48 fps video, dropping every other frame.  I have seen this done where people have used the 48 fps recording on GoPro cameras.

B or B+

Extract

The Raspberry Pi Model B+ has recently been released with improved USB hardware.  This test aimed to determine if this new hardware would reduce the latency compared to the Model B.  And the answer I’m afraid is No.

FPiV Model B v B+

Setup

The new USB hardware requires different firmware from previously.  As I was going to have to rebuild the operating system anyway, I opted to try a minimal installation using raspbian-ua-netinst.  There was a second reason for doing this, which was to install the “Jessie” build of Raspbian rather than “Wheezy” because “Jessie” comes with the latest build of gstreamer as standard.  This is currently at version 1.4.1.  In “Wheezy” you only have access to version 1.0.

Two installs were completed using identical procedures. One on my old model B and one on a new B+.

The test environment was set up the same as previously, with the camera pointing at the laptop screen.  The camera was positioned so that at least ten iterations of the image was visible at once. I used the LED on my phone with a flashlight app for measurement indicator.  The tests were run with no overclocking.

I used a UDP stream using the same scripts as for the previous Transport Stream and Overclocking Test.

A GoPro 3 Black was used to record the screen at 120 fps.

Three identical tests were run.  One for each Raspberry Pi using the operating system that had been built on it and then a third test using the B+ micro SD card in the model B with an adapter.  A number of led on/led off cycles were recorded and then two cycles from each run were picked at random for analysis.

Results

The GoPro video was analyzed using VideoReDo H264 to establish the number of frames between the LED switch on (or off) and the tenth iteration switch on/off.  This value was then divided by 1.2 to get the latency in milliseconds.

  1. Model B, Native built OS : 156ms, 150ms, 138ms, 147 ms = an average of 147ms +/- 10ms
  2. Model B+, Native built OS : 153ms, 154ms, 157ms, 154ms = an average of 154 ms +/- 3ms
  3. Model B with B+ built OS : – 148ms, 142ms, 158ms, 148ms = an average of 149 ms +/- 8 ms

Conclusion

For now the Model B+ appears to give a slightly higher latency than the Model B.  This is a shame as the B+ has a much cleaner layout.  Hopefully, the latency will improve with time as the USB drivers are optimised.  The “Jessie” build of Raspbian has not been officially released yet.

The results for the Model B were slightly better than the previous Transport Stream and Overclocking Test by about 10 millseconds so it appears the minimal raspbian-ua-netinst build was worthwhile.  See this post for details on how to configure raspbian-ua-netinst for “Jessie”.

Transport Stream and Overclocking Comparison

Extract

Various issues have kept me grounded for a couple of months, but I have managed to get some bench testing done.  Two things I wanted to test were a comparison of TCP versus UDP for the transport stream and the effect of overclocking the Pi.  In both cases the results were disappointing with only minimal reductions in overall latency.

TransportAndOverclockLatency

Setup

The test environment was set up the same as previously, with the camera pointing at the laptop screen.  The camera was positioned so that at least ten iterations of the image was visible at once. I used the LED on my phone with a flashlight app for measurement indicator.  The tests were run with the no overclocking and then the maximum (Turbo) setting available in raspi-config.

The scripts for transmitting and receiving the video stream are shown below.

A GoPro 3 Black was used to record the screen at 120 fps.

TCP – Pi Transmitter Script

#!/bin/sh
cat $0

# Zero duration is continuous
DURATION=0
WIDTH=1280
HEIGHT=720
FRAMERATE=30
BITRATE=2500000
IP=$(ip -o addr show wlan0 | sed -n 's/.*inet \([0-9.]*\)\/.. .*/\1/p')
PORT=5000

/opt/vc/bin/raspivid -t $DURATION -w $WIDTH -h $HEIGHT -fps $FRAMERATE -b $BITRATE -n -o - | gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay config-interval=1 pt=96 ! gdppay ! tcpserversink host=$IP port=$PORT

TCP – Laptop Receiver and Recording Script

#!/bin/bash
source ./settings.conf

NOW=`date +%Y%m%d%H%M%S`
FILENAME=$NOW-Rx.ts

gst-launch-1.0 -v tcpclientsrc host=$IP port=$TCPPORT ! \
  gdpdepay ! rtph264depay ! \
  h264parse config-interval=96 ! \
  tee name=t ! queue ! \
  video/x-h264, framerate=25/1 ! avdec_h264 ! videoconvert ! autovideosink sync=false t. ! \
  queue ! mpegtsmux ! filesink location=$FILENAME

UDP – Pi Transmitter Script

#!/bin/sh
cat $0

# Zero duration is continuous
DURATION=0
WIDTH=1280
HEIGHT=720
FRAMERATE=48
BITRATE=2500000
IP=$(ip -o addr show wlan0 | sed -n 's/.*inet \([0-9.]*\)\/.. .*/\1/p')

#Edit the address below to match the receiving computer
TARGETIP=192.168.42.26

UDPPORT=5001
NOW=`date +%Y%m%d%H%M%S`
FILENAME=$NOW-Tx.h264

/opt/vc/bin/raspivid -t $DURATION -w $WIDTH -h $HEIGHT -fps $FRAMERATE -b $BITRATE -n -o - | gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay config-interval=1 pt=96 ! udpsink host=$TARGETIP port=$UDPPORT

UDP – Laptop Receiver and Recording Script

#!/bin/bash
source ./settings.conf

NOW=`date +%Y%m%d%H%M%S`
FILENAME=$NOW-Rx.ts

gst-launch-1.0 -v udpsrc port=$UDPPORT \
  caps='application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264' ! \
  rtph264depay ! h264parse config-interval=96 ! \
  tee name=t ! queue ! \
  video/x-h264, framerate=48/1 ! avdec_h264 ! videoconvert ! autovideosink sync=false t. ! \
  queue ! mpegtsmux ! filesink location=$FILENAME

settings.conf

#!/bin/bash
export IP=192.168.42.1
export TCPPORT=5000
export UDPPORT=5001
export FRAMERATE=25

Results

The GoPro video was analyzed in Sony Movie Studio 11 to establish the number of frames between the LED switch on and the tenth iteration switch on.  This value was then divided by 120 to get the number of seconds for ten iterations, divided by 10 to get the number of seconds per iteration and finally multiplied by 1000 to get the latency in milliseconds.

  1. TCP – No Overclock – 160 ms
  2. TCP – Turbo Overclock – 160 ms
  3. UDP – No Overclock – 157 ms
  4. UDP – Turbo Overclock – 155 ms

As you can see from the graph, there is very little difference between the values. Switching to UDP gives a 3ms reduction in latency over TCP and the Turbo overclock removes another 2ms.  TCP doesn’t seem to benefit from overclocking at all.

Conclusion

Based on these results, I plan to switch future testing to UDP.  I’m not going to bother overclocking though, prefering the lower current draw and better long term stability of no overclocking.

Raspberry FPiV – Flight 4 – 1, 2, Tree

Extract

The aim of this flight was to test the airframe and the electronics after the unplanned lawn dart into the mud.  It was also the first test of the CRIUS CN-06 V2 U-blox GPS module.  I didn’t bother to stream the video to the laptop.  Instead I used the JuiceSSH app on my phone to access the Raspberry Pi and start a local 1080p HD and GPS recording.

Setup

The Crius GPS cable terminates in a standard 0.1″ pitch, four pin socket.  The pin outs on the Raspberry Pi are spread out over five pins, so I borrowed a single socket housing from a PCB development jumper and re-arranged the connections as shown in the photo below.

2014-05-23 13.11.08

The GPS was held on to the case with BluTak.

The only other change was to mount the camera on an extension so I could point it further down and to the right.  The aim being to get the propeller out of shot.

I didn’t plan for any big tests on this flight so didn’t bother setting up the laptop.  Instead I used the JuiceSSH app on my phone to start recording the GPS and video output on the Pi using a short script.  As I wasn’t streaming the video, I bumped the resolution up to full 1080p.

#!/bin/sh
cat $0

# Zero duration is continuous
DURATION=0
WIDTH=1920
HEIGHT=1080
FRAMERATE=25
BITRATE=10000000
NOW=`date +%Y%m%d%H%M%S`
FILENAME=$NOW-Pi.h264
gpxlogger -i 2 -f $NOW-Pi.gpx &

/opt/vc/bin/raspivid -t $DURATION -w $WIDTH -h $HEIGHT -fps $FRAMERATE -b $BITRATE -n -o $FILENAME

The Flight

Almost immediately I felt something wasn’t working properly.  Whilst the model was flying at normal speed, the climb rate was very low.  The wind wasn’t as strong as previously, but there still seemed to be some significant turbulence tossing the plane around.  After only one and a half circuits it started descending whilst over the far boundary and landed in a tree.

PlaneStuckInATree

The only visible damage was a broken tailplane and a broken prop.  I still had a WiFi connection from my phone to the Pi and was able to stop the recording and shut it down.  I had a 4m long strap in the car with a heavy buckle on one end, but it wasn’t long enough to reach the plane.  As this was lunchtime, I eventually had to leave it up the tree and return to work.

After work I stopped in at a local hardware store and bought 15m of chord and a pack of 10mm shackles.  The plane was still up the tree when I got back to the field, so I tied two of the shackles chord and started launching it skywards.  I soon found out I needed to tie some shackles to the loose end to stop the whole lot ending up in the tree.  After about 10 minutes, I managed to get the chord over the fuselage and pull it free of the branches.

After disconnecting the battery, a more detailed examination revealed the only other damage as a dent in the nose.  As the ESC had been connected for five hours, the battery had continued to drain and when checked was well past the minimum voltage.  One of the cells was reading 2.15V, so that battery will be going for recycling.

Post-Mortem

I was able to extract the video and gps files from the SD card.  It turns out the propeller was still partly visible in the video and because of this I could see the low voltage cutout had activated just as the plane crossed the far boundary. This is why it started descending and ended up in the tree.  Why the battery went flat so quickly was another matter, as was the lack of power and the inability to climb.

Once I got everything on the bench I found the motor felt a bit rough.  I can only assume that grit had got into the bearings after the mud bath.  This could account for the increased current draw with the reduced power output.

Results

I think pointing the camera off to the side was a bad idea, even for a non FPV flight.

The one positive result was the CRIUS GPS.  There was an excellent correlation between the video and the GPS track, as the two pictures below show.

Flight4-PowerLine

Flight4-Trees

Additional benefits for the CRIUS are:

  • Doesn’t require a separate power supply as it doesn’t draw too much power from the Raspberry Pi.
  • It’s a compact all-in-one unit.
  • Freely available from eBay.

Conclusion

From now I shall be using the CRIUS GPS.

As the aircraft needs some rebuilding, I’m going to rework the internals so that I can mount the Raspberry Pi internally, with the camera above the nose.  I also need to build a new power pod the will mount above the wing.

As an alternative platform for some actual First Person View flying, I’m also building a tricopter.

Raspberry FPiV – Flight 3 – Recording on the Ground

Extract

After much trawling through the forums, I found a way to record the video stream at the receiving end using gst-launch-1.0 in such a way that it can be converted to a playable format later on without re-encoding and the resulting loss of quality.  Eventually, replacing gst-launch-1.0 with a dedicated gstreamer application would allow a valid video file to be recorded straight from the stream without the current post processing. This flight also included a test of the external amplified antenna for the Ultimate GPS Breakout board. The results were disappointing, not attaining sufficient accuracy for the desired application. The flight was cut short due to a combination of high, turbulent wind and dumb thumbs, but was long enough to get meaningful results. You need to watch the video at 720p for a meaningful comparison. Please bear in mind that the video has been re-encoded at least once, possibly three times between the Pi and your browser. https://www.youtube.com/watch?v=eCazn-JbR_0

Hardware Setup

The amplified GPS antenna comes with 5 metres of cable and a heavy magnet.  I shortened the cable and removed the magnet, but the parts were still unwieldy when strapped to the top of the plane.  Hardware wise this was the only addition to the previous test. 2014-04-25 13.15.03

 Software Setup

I had already generated and swapped ssh keys between the Pi and the laptop to remove the need to enter a password when connecting. While I’ve been testing different GPS setups , I’ve started gpsd using a small shell script

#!/bin/bash
echo Starting gpsd
sudo gpsd /dev/ttyAMA0 -n -F /var/run/gpsd.sock

I want to use the current time and date for the video and gpx file names.  As the Pi does not have a real time clock, I set the time from GPS using a  script based on one I found at  http://blog.petrilopia.net

#!/bin/bash
sudo date -s '01/01/2014 00:01'
sleep 1
GPSDATE=`gpspipe -w | head -10 | grep TPV | sed -r 's/.*"time":"([^"]*)".*/\1/' | head -1`
echo $GPSDATE
sudo date -s "$GPSDATE"

Next I run cgps to check the GPS accuracy.  Once it stops improving I exit cgps and run the third script to start gps logging and record the video stream.  The video is recorded as the raw h264 stream with a Tx (Transmitter) suffix.

#!/bin/sh
cat $0

# Duration in milliseconds. Use 0(Zero) for continuous
DURATION=1800000
WIDTH=1280
HEIGHT=720
FRAMERATE=25
BITRATE=2500000
NOW=`date +%Y%m%d%H%M%S`
FILENAME=$NOW-Tx.h264
gpxlogger -i 2 -f $NOW.gpx &
IP=$(ip -o addr show wlan0 | sed -n 's/.*inet \([0-9.]*\)\/.. .*/\1/p')
PORT=5000

/opt/vc/bin/raspivid -t $DURATION -w $WIDTH -h $HEIGHT -fps $FRAMERATE -b $BITRATE -n -o - | tee $FILENAME | gst-launch-1.0 -v fdsrc ! h264parse ! rtph264pay config-interval=1 pt=96 ! gdppay ! tcpserversink host=$IP port=$PORT

Finally, back on the laptop, in a new terminal window, I run a script to capture and display the video stream.  The h264 file is multiplexed (muxed) into an mpeg transport stream (ts) container with a Rx (Receiver) suffix.

#!/bin/bash
# Read the Settings file
source ./settings.conf

# Set file name from current time
NOW=`date +%Y%m%d%H%M%S`
FILENAME=$NOW-Rx.ts

# Read the stream.
gst-launch-1.0 -v tcpclientsrc host=$IP port=$PORT ! \
 gdpdepay ! rtph264depay ! \
 h264parse config-interval=96 ! \
 tee name=t ! queue ! \
 video/x-h264, framerate=25/1 ! avdec_h264 ! videoconvert ! autovideosink sync=false t. ! \
 queue ! mpegtsmux ! filesink location=$FILENAME

# Try to copy the file from SD card to Hard Disk
if [ ! -d $TARGETDIR ]; then
 sudo mount /dev/sdb4 /mnt
fi
cp $FILE $TARGETDIR/$FILENAME

Here is a sample settings.conf

#!/bin/bash
export IP=192.168.42.1
export PORT=5000
export FRAMERATE=25
export TARGETDIR=/mnt/home/user

The Flight

When I arrived at the field the wind felt to be blowing at a constant 10 mph.  In the short time it took to get everything ready it had started gusting much harder.  Having driven to the field and got everything setup I was determined to fly so opened the throttle and gave it a good chuck. My first realisation was that the wind was stronger than anything I had flown in before.  In order to get the aircraft to track up the field and not be blown out the back, I had to point it 20 to 30 degrees past the desired direction of flight.  Also, as the aircraft got further up the field and closer to the trees, the turbulence made it harder to control.  Eventually, it got bucked around to the point I lost orientation, dumb thumbed the rudder flipping it upside down and I wasn’t able to recover it before it hit the ground. When I checked later, the met office were quoting 26 mph gusts.

The Aftermath

The previous months rain had softened the ground enough that the aircraft nose was stuck in three or four inches with no apparent damage.  The rubber bands had separated, releasing the wings.  Any electronics that weren’t screwed down were scattered around the crash site and given a liberal coating of mud. Everything was dismantled and cleaned in fresh water, then left to dry.   When I did reconnect everything, it all worked.  The only problem was that camera now has a green tint.  I suspect that that the mud has abraded the lens coating, either on impact or during cleaning.  As they are cheap I bought a new one.

Results

I was worried that the crash would have lost or corrupted the on-board video recording.  As it turned out, only the last few seconds of the flight were lost, as you can see in the video.  After collecting the scattered components, I found the X terminal on the laptop reporting that the SD card had become read only.  It was still receiving and recording a video stream, although this was mostly a totally brown frame which turned to a greeny orange when I removed most of the mud.

GPS

I’ll cover the GPS first as it’s going to be short. The results using the larger amplified antenna were better, but still not good enough.  There seems to be either a lag or an offset in the computation.  The two images below show the problem.  For the next flight I’m going to use the CRIUS CN-06 V2 U-blox GPS module. Flight3-CrashSite Flight3-CricketSquare

Video 

The video recorded on the Pi (Tx) was copied to a Windows PC using WinSCP.  Then I used MP4BoxGUI to mux it into a MP4 file.  I have not had any problems using the generated file. With the video recorded on the laptop (Rx) as a transport stream, the first task was to convert it into an mp4. The following script takes the file name as an argument and creates two new videos.  This need to be run on a machine using Gstreamer version 1.2 (Check using gst-launch-1.0 –version).

#!/bin/bash
echo .
echo Remux
echo .
gst-launch-1.0 -v filesrc location=$1 ! tsdemux ! h264parse ! video/x-h264,framerate=25/1 ! mp4mux ! filesink location=$1.remux.mp4
echo .
echo Re-encode
echo .
gst-launch-1.0 -v filesrc location=$1 ! tsdemux ! h264parse ! video/x-h264,framerate=25/1 ! avdec_h264 ! videoconvert ! x264enc bitrate=2500 ! mp4mux ! filesink location=$1.reenc.mp4 

The first gstreamer pipeline (Remux), simply extracts the h264 video from the transport stream container and them multiplexes it into an mp4 container.  The h264 stream is unchanged, so there is no loss of video quality.  To make this file viewable on Windows required a couple of extra steps.  With the file copied on to the Windows PC I used MP4BoxGUI to demux and extract the h264 stream (again) and then to Mux it back into an MP4 file.  This new file was playable in Windows media Player. The second gstreamer pipeline (Re-encode), re-encodes the video as a new h264 file and multiplexes it into an mp4 container.  While re-encoding has to reduce the picture quality, it is not obvious without close inspection.  I use the re-encoded video in Sony Movie Studio 11.0, which had trouble reading the first remuxed file.  I also had to manually set the frame rate in Sony Movie Studio 11 to 25 fps as by default it was using 29.97 fps making it impossible to sync the Tx and Rx videos together.

Conclusion

The video posted on Youtube shows the Tx and Rx videos side by side.  They are fully synced and without a gap between I don’t think you can spot a join or see any difference in video quality.  Although a bit of a faff, for now I have a working pipeline to record usable video on the ground. As for the GPS, the search goes on.

Improved Raspberry Pi Power Supply

Extract

I wanted to improve the power supply for the Raspberry Pi to remove all the adapter cables and to allow the Pi to be powered without arming the motor.  These aims have been achieved by adding a USB socket and a LiPo balance connector socket to the existing Battery Eliminator Circuit (BEC).

2014-04-02 08.14.58

Power Input

I cut the socket from a 3S LiPo balance connector extension and soldered the two outer wires to the BEC input.   Although the two centre wires are not used, I left them in place to provide some strain relief when unplugging the socket from the battery.  With the BEC powered from the balance connector, the Pi can be powered independently of the flight motor, but still from the same battery.

Power Output

I wanted to use a right angle Micro-B USB cable to connect the power to the raspberry Pi.  The original straight plug stuck out too far and the bare wires were not very sturdy.  To use this I soldered a USB A Socket to some copper strip board. The middle two data pins do not line up with the 0.1 inch hole spacing so needed a little adjustment to fit the board.  I also needed to enlarge the holes to take the sockets fixing legs.  The board was left long and tapered to match the width of the BEC.

The BEC output wires were cut and soldered to the 5V and ground tracks on the board.  I kept the servo connector to power the GPS boards separately from the Pi to bypass the Pi’s internal power limitations.

2014-04-02 08.17.29

2014-04-02 08.16.57

Everything was lined up as compact as possible and then bound together using heat-shrink tube.

2014-04-02 08.17.54

The final item turned out neat and compact.

2014-04-02 08.15.13

The stripboard provides a nice flat surface for some velco to attach the unit inside the aircraft.  The 30 cm  USB cable is just long enough to reach the Raspberry Pi on top of the wing.

Denied!

For some reason my Raspberry Pi has stopped working at the field.  Everything is fine on the bench, but as soon as it’s strapped to the the plane, the Laptop can’t connect. Without a monitor and keyboard at the field, I can’t diagnose the problem.

I have two thoughts on the matter.

1. The case and board are distorting due to the rubber bands, preventing the SD card from getting a full electrical connection.  I’ve made a platform for it to sit on which should hold everything flat.

2. Although the Pi is powered from a 3 Amp BEC, there is an internal limit to how much current it can supply to peripherals.  I have added the external amplified antenna to the Ultimate GPS Breakout and it maybe the extra current draw is causing problems.

To remedy this I have modified the wiring harness so that it can be powered directly.

While on the subject of the amplified antenna, I have also shortened the cable from its original 5 metres to something more manageable and removed the heavy permanent magnet from the housing.

Hopefully these measures should get things working again.

Image

Raspberry FPiV – Flight 2 (Line of Sight)

 Extract

Only a couple of days after my first flight, we had another sunny day, so I took the opportunity to go flying.  I hadn’t changed the setup from the first flight, but wanted to try pushing the range out further.  I found after the flight that the GPS had never achieved a lock and no track was recorded.  Reviewing the on-board camera footage, I found some clear landmarks that could be used to calculate a new maximum range of 350 metres.  This is about as far as I can fly line of sight with this airframe.

While the flight was performed line of sight, I did manage a couple of glances at the laptop screen with the headcam to show the stream was still running.

Setup

The computer and aircraft setup was the same as for the first flight.  I pointed the camera down a bit further so more ground and less sky was recorded.  Unfortunately it also meant more propeller was recorded.  I did move the rudder push rod in one hole on the servo to reduce its authority under power.  My old Futaba Attack 4 channel radio doesn’t have dual rates.

The Flight

While still blustery, the wind was not as strong as for the first flight.  I was able to push the distance out further and a couple of quick glances at the laptop showed the stream was still running.  Towards the end of the flight, I tried a couple of loops and then flying through the (bigger) gaps in the trees.  You can’t hear it on the video, but I did clip a couple of small branches.

Flight2-Approach

Results

After the flight it was disappointing to find that the gps had never got a lock and no track had been recorded. Luckily, the on-board camera had recorded two very distinctive landmarks during a turn which could be used to triangulate it’s position.

The first was a hedge running south.

Flight2-Hedge

The second was the line of trees next to the playing fields.

Flight2-Trees

Locating these landmarks in Google Earth allowed me to measure the maximum range as 350 metres.

Flight2-MaximumRange

This distance is about my limit for line of sight flying with this airframe.

While editing the video I became aware that the Raspberry Pi camera and the GoPro headcam were running at slightly different rates.  I had synced the start of the two videos streams at the point the propeller started turning to within one frame.  After the six minute flight the Raspberry Pi was about one second behind the GoPro.  You can see this as the plane passes the last tree.

While not a big deal it does make it a pain to edit the two streams in parallel.

Conclusion

While not a totally successful flight, I was able to establish a new maximum streaming range of 350 metres.  The next step is to work out a way to record the stream at the laptop end.

Raspberry FPiV – Flight 1 (Line of Sight)

Extract

Having established from my tests with gstreamer that a low latency video stream over WiFi could be achieved from the Raspberry Pi and that a usable range could be achieved,  I decided to mount the system on an aircraft for some line of sight testing.  In order to extend the range test, I added an Adafruit Ultimate GPS Breakout Board.

The results were mixed: the video stream remained consistent throughout the flight although with higher latency than expected and the GPS accuracy was poor.   The reasons for the problems have been identified and fixed, so the overall results are still promising.

Computing Setup

To save booting from a thumb drive every time, I resized the Windows partition on my laptop and installed Lubuntu to the end of the hard disk.  This was fine for a while, but then an update to gstreamer made it impossible to install the bad and ugly plugin packages together.  In the end I gave up on Lubuntu and installed Arch Linux on both the Laptop and the RPi.  The driving force for this was that version 1.2 of gstreamer was available on both platforms.  Having never used Arch Linux before it took me most of the Christmas holidays to get everything running properly.

To save a bit of weight I replaced the Raspberry Pi model B with a model A.  The SD card can be moved between the two boards without changes.  An  Adafruit Ultimate GPS Breakout board was added to extend the range testing.  This was connected to the Pi’s UART pins.  I didn’t solder on the battery holder for the real time clock.

The board was mounted in a bright raspberry pink case, so it would be easier to find should it become detached from the plane.  I trimmed the plastic around the SD card slot so the camera cable could be run out above the card.  The camera was mounted in a PiBow holder and bolted to the case.  As the PiBow holder clips together, I glued the joints to secure everything.

Finally, I created some scripts to get everything configured and talking.

Aircraft Setup

I retrieved my old “Sorcerer” slope soaring trainer from the loft.  This design dates from 1969, although mine was built during the eighties.  It’s not pretty, but it is tough.  The radio system was a Futaba Attack 4 channel system on 35MHz and so wouldn’t interfere with the 2.4GHz WiFi signal.  As it hadn’t been used for 25-30 years, I replaced the transmitter battery, but apart from that everything worked fine.

As this was a glider, I built a power pod from birch and lite plywood to sit in place of the hatch.  Having the motor up front put the centre of gravity in the right place without adding additional weight.  The flight battery was replaced with a 1300 mAh LiPo which powered the receiver and servos through the ESC’s Battery Eliminator Circuit (BEC). I used a propeller that had previously been used on a multirotor that was well balanced.

2014-01-10 13.28.19

The Pi was powered by a dedicated 5V 3A BEC connected to the LiPo battery’s balance connector.  Everything was held on with rubber bands.

2014-01-10 13.28.28

The Flight

The first launch turned into a powered glide.  I hadn’t allowed enough throttle for the extra weight and drag.   The second launch had it climbing and zig-zagging up the field.  The Sorcerer has a huge rudder on a fin that is little more than a post.  With the added prop wash over the rudder, turbulence from the everything strapped on top of the wing and the gusting wind, it was difficult to keep it straight and smooth.  Once some height had been achieved and the throttle reduced everything settled down. I flew for about 5 minutes doing big circuits of our local playing field, before bringing it in to land.

Flight1-ScreenSetup

Results

As I was flying without a spotter, I didn’t look at the laptop screen until the flight was finished.

After retrieving the glider, I found the stream was still running which was a good start.  I hadn’t checked the latency with the new system before the flight, so it was disappointing to find it was running noticeably higher than I had achieved in my earlier tests.  I still had the original Raspian SD card and Lubuntu Thumb drive and a quick check showed that that setup still ran with low latency.  Research suggests the higher latency is due to incompatibilities between Arch Linux and the particular Intel chipset in the laptop.  It could also be that the hard drive speed is slower than the thumb drive, especially as the linux partition is at the end of the available drive space.

The next check was the GPS track.   I used WinSCP to get the video and gpx files off the RPi.  I found a really useful website that animated the track in real-time: www.gpsanimator.com/StarTraX.   I did a video capture from the screen using Applian Replay Video Capture which I then synced with the on-board and head camera footage in Sony Movie Studio 11.  As well as producing the video, this also allowed me to get frame grabs to calculate the gps error.

The animation showed the track jumping all over the place.   If it is to be believed, most of the flight was spent on the wrong side of the trees and I was flying behind where I was standing.  When synchronized with the video it is easy to see the inaccuracy.

Here is an example.  This image shows the glider crossing left to right in front of me.  In the bottom right hand corner is a tree.

Flight1-GPS1

 

Now look at the GPS track. I’ve marked on where I was stood, the position of the tree and the observed track.  The recorded track shows aircraft moving right to left, well behind me and over a small wood. The error is about 60 metres.

Flight1-GPS3

 

An even clearer example is the landing.  The error is about 130 metres.  Seeing as the current maximum tested range is only 240 metres, this is not good.  After being stationary for a little while, the GPS eventually caught up.

 

 

 

Flight1-Landing

I’ve got a Crius CN-06 V2 U-Blox based GPS which has a much larger antenna as the picture below demonstrates.  I also have the high gain remote antenna for the Ultimate GPS breakout, which is the same size as the Crius.  Finally, I’m going to add the real time clock battery holder, which should improve the warm start times.  At least one of these should improve things. 

 

 

GPS Size

 

Conclusion

While there were issues with the latency and the GPS, as a first flight the main purpose was achieved.  The streaming from air to ground worked and remained stable throughout the flight.  There is certainly enough promise to keep the project moving forward.