Reinforcement Learning for Autonomous Dynamic Soaring in Shear Winds

Corey Montella and John Spletzer

Abstract—Dynamic soaring (DS) is an aerobatic maneuver whereby a gliding aircraft harnesses energy from horizontal wind that varies in strength and/or direction to support flight. Typical approaches to dynamic soaring in autonomous unmanned aerial vehicles (UAVs) use nonlinear optimizers to generate energy-gaining trajectories, which are then followed using traditional controllers. The effectiveness of such a strategy is limited by both the local optimality of the generated trajectory, as well as controller tracking errors. In this paper, we investigate a reinforcement learning (RL) approach working in continuous space to control a DS aircraft flying in shear wind conditions. The RL controller operates in two stages: In the first stage, it observes a traditional sample-based controller flying a locally optimal DS trajectory generated a priori. In the second stage, the sample-based controller is removed and authority is passed to the RL algorithm. We show that by deviating from the original planned trajectory, the RL controller is able to achieve better performance than its baseline teacher controller.

To appear at the 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems

Download the full text here.

Closing the Loop in Dynamic Soaring

John J. Bird, Jack W. Langelaan, Corey Montella, John Spletzer, and Joachim Grenestedt

Abstract – This paper examines closed-loop dynamic soaring by small autonomous aircraft. Wind field estimation, trajectory planning, and path-following control are integrated into a system to enable dynamic soaring. The control architecture is described, performance of components of the architecture is assessed in Monte Carlo simulation, and the trajectory constraints imposed by existing hardware are described. Hardware in the loop simulation using a Piccolo SL autopilot module are used to examine the feasibility of dynamic soaring in the shear layer behind a ridge, and the limitations of the system are described. Results show that even with imperfect path following dynamic soaring is possible with currently existing hardware. The effect of turbulence is assessed through the addition of Dryden turbulence in the simulation environment.

To appear at the 2014 AIAA Science and Technology Forum and Exposition

Download the full text here.

Dynamic Soaring Featured in Popular Science

Our Dynamic Soaring project was featured in the latest issue of Popular Science magazine!

The Future Of Flight: Planes That Never Need To Land

Story by: David Hambling

For decades, work on dynamic soaring progressed slowly. Radio-control glider operators took advantage of the technique to extend their flights, but scientists didn’t know if they could apply it to larger craft. Then in 2006, a team from the U.S. Air Force and NASA flew a modified L-23 Blanik sailplane over Edwards Air Force Base, proving that a large craft is capable of dynamic soaring maneuvers.

Now a team at Lehigh University, led by engineering professor Joachim Grenestedt, is refining the concept…

Read more at Popular Science

Dynamic Soaring Featured in Resolve Magazine

Read about the Dynamic Soaring project in Resolve Magazine, published by the P.C. Rossin College of Engineering and Applied Sciences at Lehigh University.

Endless and Effortless Flight – Inspired by the Albatross, Researchers Set Their Sights on the Jet Stream

Story by: Chris Quirk
Photography by: Ryan Hulvat

The inspiration was simple: Create a small unmanned glider that once launched would fly on its own… indefinitely.

Nothing, however, can fly forever. The unflagging forces of gravity, drag, and air friction all conspire to erode the strength of the airborne and pull them to ground. How close — given the relentless opposing factors — could you get to a plane that would fly underpowered and unassisted for extended periods of time?

While the concept of constant flight might seem outlandish, this is the modest proposition that Lehigh professors Joachim Grenestedt and John Spletzer have set for themselves.

Read the rest at

JetStreamer Wing Featured on Composite World

University professors pursue perpetual flight with composites

Lehigh University (Lehigh, Pa., USA) reports that one of its engineers has developed a carbon fiber composite wing for use on an unmanned aircraft being designed for high-altitude perpetual flight.

The first uniquely designed carbon fiber wing has emerged from Lehigh’s Composites Lab. The 6.5m/21.3-ft wing was made in a single molding process, complete with wing planks, spar caps to fortify the wings, six internal webs to carry shear loads and a trailing edge ready to accommodate wing flaps….

Read more at Composite World

Dynamic Soaring in Hurricanes

Joachim Grenestedt, Corey Montella, and John Spletzer

Abstract – The potential for a gliding UAV to sustain flight by dynamically soaring in a hurricane is investigated. Leveraging extensive storm observations, the wind profile of the hurricane eye is modeled as a continuous function that is zero at the center and increases as a power of the radius. We then derive the equations of motion for a UAV flying in this wind field, and prove analytically that if the wind field exponent n = 1, dynamic soaring is not possible. This analytical result is also validated in simulation. We also provide extensive simulation results for the case where the wind field exponent n > 1. These results indicate that dynamic soaring is in fact possible for such storms, and the velocity gain for a single dynamic soaring cycle is correlated with the wind field gradient.

To appear at the 2012 International Conference on Unmanned Aircraft Systems (ICUAS)

Download the full text here.

Wind Field Estimation for Autonomous Dynamic Soaring

Jack W. Langelaan, John Spletzer, Corey Montella, Joachim Grenestedt

Abstract — A method for distributed parameter estimation of a previously unknown wind field is described. The application is dynamic soaring for small unmanned air vehicles, which severely constrains available computing while simultaneously requiring updates that are fast compared with a typical dynamic soaring cycle. A polynomial parameterization of the wind field is used, allowing implementation of a linear Kalman filter for parameter estimation. Results of Monte Carlo simulations show the effectiveness of the approach. In addition, in-flight measurements of wind speeds are compared with data obtained from video tracking of balloon launches to assess the accuracy of wind field estimates obtained using commercial autopilot modules.

To appear at the 2012 IEEE International Conference on Robotics and Automation (ICRA)

Download the full text here

Towards Perpetual Flight of a Gliding Unmanned Aerial Vehicle in the Jet Stream

Joachim Grenestedt and John Spletzer

Abstract – In this paper, we investigate the potential of a self-powered gliding aircraft to remain aloft indefinitely. We focus specifically on operations in the jet stream where persistent wind gradients can be used to enable unpowered flight. The advantage of such a paradigm is that power requirements are reduced to those needed for aircraft navigation, control, and communication systems. To investigate the feasibility of such an approach, we examine both aircraft design requirements for perpetual flight, as well as trajectory strategies for solar power generation. Both the aircraft design and power generation problems are cast as non-linear optimization problems where the aircraft equations of motion and periodicity serve as constraints to ensure that sustainable flight trajectories are maintained. Simulation results indicate a suitable aircraft design capable of unpowered flight in the jet stream and capable of generating sufficient electrical power can be achieved.

To appear at the 2010 IEEE International Conference on Decision and Control.

Read the full text here.