[Canberrauav] safety debrief of our OBC takeoff
Andrew Tridgell andrew at tridgell.net
Fri Sep 26 20:20:23 EST 2014
Hi All,
Thanks to everyone who has been passing on congratulations to us!
We will be posting more information about our OBC flight soon, but we
wanted to start off with a discussion and debrief of our takeoff, which
had a significant safety issue which we think is worth sharing
publicly. We hope by discussing the issue that other small UAV pilots
will avoid the same mistake.
Summary
-------
During the autonomous takeoff the plane came uncomfortably close to the
GCS control tent, OBC organisers and media to the eastern side of the
airfield. Our analysis shows that the root cause of the issue was a
combination of medium strength cross-wind, poor communication and the
binary nature of the safety pilots ability to abort a takeoff.
Details
-------
We started an automonous takeoff with our 2.7m VQ Porter UAV (TOW 14kg)
from the southern end of Kingaroy airport at 7:55am on Friday the 26th
of September 2014. The flight plan was for a straight down the runway
takeoff to the south, with a takeoff pitch after rotation of 12 degrees
and transition to navigation for the primary mission at 40m above the
runway.
On leaving the runway the plane drifted left in the wind, bringing it
uncomfortably close to the spectators and officials to the western side
of the runway, outside the gable markers. Our safety pilot had only a
very short time to decide if we should abort the takeoff, and decided
the safest thing to do was to continue the takeoff. Some people have
(very understandably) questioned that decision.
Flight Path
-----------
The initial part of the takeoff was routine, with the wheels leaving the
runway after about 50 meters. At this point the aircraft was 34 meters
on the runway side of the gable markers. The plane was on a heading of
134 degrees (grid) and reached an altitude of 8 meters before crossing
the gable markers. For reference, our GCS antenna mast is 4.5m tall, and
that was the closest structure to the runway. At the time we crossed the
line of the gable markers the aircraft was 27 meters south of our GCS
antenna mast (which has its own GPS, thus allowing accurate
measurement).
We estimate that the closest distance our aircraft came to our GCS mast
was approximately 25 meters (in 3 dimensions). That is closer than we
should have come for good safety.
Wind Issue
----------
The first cause of the issue was lack of awareness by the UAV GCS
operator of the wind conditions on the runway. At the time we put up our
GCS tent the wind was lower, and we planned for a zero-wind takeoff,
straight down the runway. The enclosed tent around the GCS meant that we
did not notice the rising cross-wind, which was around 12 knots from the
west on takeoff.
We should have asked the safety pilot for a wind update before takeoff,
but neglected to do so. The safety pilot also should have pro-actively
informed us of the wind, but didn't. The cause of both of those
communication issues was that in our normal operating procedures we can
see the wind sock from the tent, but for this flight the wind sock was
not visible, and the tight tent meant the wind was not obvious.
If we had been fully aware of the cross-wind we would have planned for a
cross-wind takeoff, by pointing the plane for a angled takeoff towards
the south east corner of the runway.
Abort System
------------
The second key problem with the takeoff was the binary nature of our
abort system. The safety pilot only had two choices - he could let the
aircraft continue an autonomous takeoff, or he could take full manual
control and abort the takeoff.
Neither option was appealing. The plane was clearly flying closer than
it should to the spectators, but was also clearly under good control of
the autopilot, and was clearly going to miss the spectators. We know
from experience that when a safety pilot aborts a takeoff part way
through that the first second or two of the abort results in the
aircraft having initially less control, as the pilot comes to terms with
the aircrafts movement and wind and gets the throttle right.
Under normal conditions we would flown the aircraft with STICK_MIXING
enabled, which provides the safety pilot with a middle ground between a
full takeover and continuing the mission. With STICK_MIXING enabled the
pilot could have demanded some right roll, or added right rudder to
steer the aircraft smoothly away from the spectators without
interrupting the rest of the takeoff (autonomous throttle and pitch
control in particular).
We greatly regret that we had disabled STICK_MIXING for this flight, as
we were concerned that using STICK_MIXING may have invalidated the 10
points for fully autonomous takeoff in the OBC. That was a poor
decision, and deprived our safety pilot of the best option for ensuring
a maximally safe takeoff.
Conclusion
----------
Some poor decisions and communication led to us coming closer than we
should have to spectators at the OBC 2014. We hope that the above
analysis helps other operators make better decisions, and will result in
safer flight in the future.
We would welcome criticism of the above analysis.
Andrew Tridgell (GCS operator)
CanberraUAV
[Canberrauav] safety debrief of our OBC takeoff
Chris Gough christopher.d.gough at gmail.com
Sat Sep 27 01:05:01 EST 2014
Thanks for the write-up Tridge.
The team on the ground in Kingaroy has spent some time today trying to
analyse the root-cause of the dangerous takeoff, and considering possible
opportunities for improvement. I'm going to post my personal notes and
ideas from those discussions here while it's still fresh in my mind, and
I'd like to explicitly invite anyone to comment, offer suggestions or
ideas, or ask questions (preferably direct to the list, not to me
privately).
Please consider this analysis a preliminary draft for discussion. I will
collate any feedback and present a more advanced analysis on our first
scheduled Monday night meeting after the competition (6th October), and
then we can make decisions about necessary follow-up actions at or after
that meeting.
Root Cause Analysis ("5 whys" technique)
===============================
Problem: Spectators (officials, other competitors, etc) were exposed to an
unsatisfactory risk of collision with the aircraft during takeoff.
1) Why were the spectators exposed to collision-risk?
1.a) the aircraft veered off the expected course during takeoff.
1.b) the spectators were positioned inappropriately.
If the pilot in command had believed it was not possible to take off safely
then he would not have issued a direction for the plane to take off.
Therefor, 1.b is not a root cause. However, during debrief we noted that
the arrangement at the runway today would not have been acceptable on our
usual test range (CMAC field).
For comparison, the steel fence around the pilots box at CMAC significantly
mitigates the collision-risk for the people that need to be close to the
aircraft. At CMAC, spectators are positioned further away from the runway
than the officials and competitors were today (and they are also behind a
steel fence). During testing prior to competition, the closest we
positioned the full ground station (van, tent, antenna mast etc) was in the
car park on the other side of the club house, much further from the runway
than it was today (this is significant because ground crew in the tent have
very limited viability of the runway).
Despite these observations, I think it's pretty obvious that the root cause
of exposure was the veering off course.
2) Why did the aircraft veer off the expected course?
2.a) the automated takeoff did not include cross-wind correction
2.b) a gust of wind changed the aircraft's orientation early in the run
(speed below full aerodynamic control)
2.c) the pilot did not abort takeoff
We discussed 2.c at length including possible opportunities for
improvement, but the consensus withing those present was that this was a
factor, but not the rood cause. Some people outside the team have expressed
a contrary opinion, and we are very open to discussing that further.
As Tridge already mentioned, if we had enabled the STICK_MIXING feature the
pilot would have been able to "nudge" the autonomous takeoff back on line
(this is our usual practice). In hindsight, the decision to disable that
feature (in attempt to demonstrate the maximum possible autonomy) was a
mistake, we failed to adequately consider the safety compromise we were
making.
The changed orientation by a wind gust is clearly not a root cause, because
wind gusts are normal and to be expected. We did discus some possible
improvements to our algorithms, but nothing significant to the root cause
of the incident. There seemed to be a general consensus that 2.a was the
root cause of the aircraft veering of course.
3) Why did the automated takeoff fail to compensate for the cross wind?
3.a) the ground crew did not tell it to
3.b) the system is not capable of autonomously determining that cross-wind
correction was necessary
3.b seems like an interesting topic, but since the system lacks the
relevant sensors (e.g. side-slip indicator) it's not "just a matter of
software". This is not the case when the plane is in flight (we do wind
triangle calculations based on ground speed and airspeed vectors), but when
the plane is stationary or slow moving on the ground, the autopilot is only
capable of measuring the headwind component.
Since our current system relies on the ground crew to instruct the
autopilot to make a cross wind correction for automated takeoffs, 3.a seems
like the root cause
4) Why didn't the ground crew instruct the autopilot to make a cross wind
correction?
4.b) they didn't know it was necessary
5) Why didn't the ground crew know about the cross wind?
5.a) there was no cross wind when they entered the tent, it appeared
suddenly soon after they sat down.
5.b) the pilot did not tell them about it
5.c) they did not ask the pilot about the wind
5.d) they did not visually check a wind sock
5.e) they did not digitally check a computerised weather station
5.a is discounted as root cause because suddenly appearing winds are normal
and to be expected.
5.d and 5.e are discounted because the windsock and weather station were
not available. While the airport had wind socks (of course), none were
visible from the ground control tent. Perhaps we should add a portable
windsock to our system, but will not always be practical or appropriate
therefor it's an inadequate response. Adding a weather station to our
antenna mast is an interesting idea (as per side discussion around 3.b) but
that's an opportunity for improvement, lack of weather station is not the
root cause of the event.
5.b is a procedural matter. The pilot could "push" wind speed information
to the ground crew without being asked, but a "pull" mechanism (checklist
item to request information from the pilot) might be more reliable, since
the ground crew are at a desk with the checklist and pen available whereas
the pilot has his hands on the transmitter, his eyes on the aircraft and
his mind on the broader situation. Therefor, in my opinion 5.c was the root
cause. Until such time as we have a weather station on the antenna mast or
somehow make the aircraft capable of applying it's own crosswind detection
and compensation, the lack of "request pilot's wind report" from our
pre-takeoff procedure is a defect.
Chris Gough (Safety Officer)
CanberraUAV
On Fri, Sep 26, 2014 at 8:20 PM, Andrew Tridgell
wrote:
> Hi All,
>
> Thanks to everyone who has been passing on congratulations to us!
>
> We will be posting more information about our OBC flight soon, but we
> wanted to start off with a discussion and debrief of our takeoff, which
> had a significant safety issue which we think is worth sharing
> publicly. We hope by discussing the issue that other small UAV pilots
> will avoid the same mistake.
>
> Summary
> -------
>
> During the autonomous takeoff the plane came uncomfortably close to the
> GCS control tent, OBC organisers and media to the eastern side of the
> airfield. Our analysis shows that the root cause of the issue was a
> combination of medium strength cross-wind, poor communication and the
> binary nature of the safety pilots ability to abort a takeoff.
>
> Details
> -------
>
> We started an automonous takeoff with our 2.7m VQ Porter UAV (TOW 14kg)
> from the southern end of Kingaroy airport at 7:55am on Friday the 26th
> of September 2014. The flight plan was for a straight down the runway
> takeoff to the south, with a takeoff pitch after rotation of 12 degrees
> and transition to navigation for the primary mission at 40m above the
> runway.
>
> On leaving the runway the plane drifted left in the wind, bringing it
> uncomfortably close to the spectators and officials to the western side
> of the runway, outside the gable markers. Our safety pilot had only a
> very short time to decide if we should abort the takeoff, and decided
> the safest thing to do was to continue the takeoff. Some people have
> (very understandably) questioned that decision.
>
> Flight Path
> -----------
>
> The initial part of the takeoff was routine, with the wheels leaving the
> runway after about 50 meters. At this point the aircraft was 34 meters
> on the runway side of the gable markers. The plane was on a heading of
> 134 degrees (grid) and reached an altitude of 8 meters before crossing
> the gable markers. For reference, our GCS antenna mast is 4.5m tall, and
> that was the closest structure to the runway. At the time we crossed the
> line of the gable markers the aircraft was 27 meters south of our GCS
> antenna mast (which has its own GPS, thus allowing accurate
> measurement).
>
> We estimate that the closest distance our aircraft came to our GCS mast
> was approximately 25 meters (in 3 dimensions). That is closer than we
> should have come for good safety.
>
> Wind Issue
> ----------
>
> The first cause of the issue was lack of awareness by the UAV GCS
> operator of the wind conditions on the runway. At the time we put up our
> GCS tent the wind was lower, and we planned for a zero-wind takeoff,
> straight down the runway. The enclosed tent around the GCS meant that we
> did not notice the rising cross-wind, which was around 12 knots from the
> west on takeoff.
>
> We should have asked the safety pilot for a wind update before takeoff,
> but neglected to do so. The safety pilot also should have pro-actively
> informed us of the wind, but didn't. The cause of both of those
> communication issues was that in our normal operating procedures we can
> see the wind sock from the tent, but for this flight the wind sock was
> not visible, and the tight tent meant the wind was not obvious.
>
> If we had been fully aware of the cross-wind we would have planned for a
> cross-wind takeoff, by pointing the plane for a angled takeoff towards
> the south east corner of the runway.
>
> Abort System
> ------------
>
> The second key problem with the takeoff was the binary nature of our
> abort system. The safety pilot only had two choices - he could let the
> aircraft continue an autonomous takeoff, or he could take full manual
> control and abort the takeoff.
>
> Neither option was appealing. The plane was clearly flying closer than
> it should to the spectators, but was also clearly under good control of
> the autopilot, and was clearly going to miss the spectators. We know
> from experience that when a safety pilot aborts a takeoff part way
> through that the first second or two of the abort results in the
> aircraft having initially less control, as the pilot comes to terms with
> the aircrafts movement and wind and gets the throttle right.
>
> Under normal conditions we would flown the aircraft with STICK_MIXING
> enabled, which provides the safety pilot with a middle ground between a
> full takeover and continuing the mission. With STICK_MIXING enabled the
> pilot could have demanded some right roll, or added right rudder to
> steer the aircraft smoothly away from the spectators without
> interrupting the rest of the takeoff (autonomous throttle and pitch
> control in particular).
>
> We greatly regret that we had disabled STICK_MIXING for this flight, as
> we were concerned that using STICK_MIXING may have invalidated the 10
> points for fully autonomous takeoff in the OBC. That was a poor
> decision, and deprived our safety pilot of the best option for ensuring
> a maximally safe takeoff.
>
> Conclusion
> ----------
>
> Some poor decisions and communication led to us coming closer than we
> should have to spectators at the OBC 2014. We hope that the above
> analysis helps other operators make better decisions, and will result in
> safer flight in the future.
>
> We would welcome criticism of the above analysis.
>
> Andrew Tridgell (GCS operator)
> CanberraUAV
>
>
--
.
[Canberrauav] safety debrief of our OBC takeoff
Andrew Tridgell andrew at tridgell.net
Mon Sep 29 13:11:26 EST 2014
Hi All,
I've now had time to look at the detailed dataflash logs off the microSD
card on the Porter Pixhawk, and I've found that the wind was not the
primary reason for the plane going off course on takeoff. It was in fact
very poor compass offsets.
During the takeoff the plane initially uses the "summed gyro", trying to
keep the sum of the Z gyro zero. After the AHRS reports that it has a
valid heading it then switches to a locked heading. It locks the heading
at the heading it first gets from the AHRS subsystem. In this case it
locked the heading at 133 degrees due to very poor compass offsets,
whereas the runway is 169 degrees.
This raises two points:
1) why were the compass offsets so badly off on takeoff?
2) how can the takeoff code be changed to not be sensitive to poor
compass offsets?
For the first question, I am still trying to determine that for
certain. The plane should have had the compass offsets from our last
CMAC flight loaded, but for some reason it didn't. I'll post a followup
if I work out why they had changed.
For the 2nd question, I'm working on a change to the takeoff navigation
code to use a summed gyro for heading hold for the whole takeoff, rather
than switching to the AHRS yaw. That should make the takeoff completely
insensitive to bad compass offsets.
Cheers, Tridge
Copyright 2014 http://canberrauav.org.au/