For a small flying insect, correcting unplanned course perturbations is essential for navigating through the world. Visual course control relies on estimating optic flow patterns which, in flies, are encoded by interneurons of the third optic ganglion. However, the rules that translate optic flow into flight motor commands remain poorly understood. Here, we measured the temporal dynamics of optomotor responses in tethered flies to optic flow fields about three cardinal axes.
For each condition, we used white noise analysis to determine the optimal linear filters linking optic flow to the sum and difference of left and right wing beat amplitudes. In the remaining case of yaw optic flow we improved predictability by measuring individual flies, which also allowed us to analyze the variability of optomotor responses within a population. Finally, the linear filters at least partly explain the optomotor responses to superimposed and decomposed compound flow fields.
Several other aspects of our experimental method impose limitations while also providing exciting avenues for future research. Nonlinearities contained in natural scenes composed of compound patterns of flow may be asymmetrical and include motion that is independent of the fly's locomotion. It will be important to expand the present analysis to more complex optic flow regimes. Next, although the optical wingbeat analyzer takes very reliable measures in real time, it measures only a two-dimensional projection of a complicated multi-dimensional wing stroke (Fry et al., 2005).
Nevertheless, our results suggest that the projections themselves are consistent for a given optic flow stimulus. Our analyses were performed entirely under open-loop conditions in which the flies had no control over image motion. Closed-loop experiments with this method are more challenging but might alter the response dynamics by putting the fly in an alternative behavioral state. During free flight these additional inputs certainly interact with the visual dynamical wing beat response. We simulated translational and rotational motions along each cardinal axis aligned along the fly's body. The stimulus was generated by simulating the perturbation of the fly's position within a cloud of randomly distributed dots .
As the pattern advanced and retreated, the projection of dots onto a cylindrical surface was computed and the corresponding LEDs turned on . Arrows show the velocity of any point based on its location in the field, however, arrow lengths are approximate for the translational velocity, since 'nearer' virtual points produce faster angular speeds. An impulse function modulated yaw by advancing the pattern one pixel (3.75 deg.) to the right.
The top plot illustrates impulses at 0 and 100 ms. The black trace below shows the difference in left and right wing beat amplitudes, the attempted turning response, to the 0 ms impulse. The response to the same increment at 100 ms, overlaid in red by the top 0 ms response shifted by 100 ms. Response to a 2 pixel increment (7.5 deg.), overlaid in red by the double top 0 ms response. The response to a pattern incremented at both 0 and 100 ms, overlaid in red by the sum of the responses to 0 and 100 ms independent increments. Scale bars, 100 ms horizontally and 0.02 V vertically; each trace is the mean of responses from 364 flies.
The top plot shows a partial m-sequence that modulated visual yaw by advancing and reversing rightward motion. The impulse response estimated from this single fly after the full 20 s m-sequence. Impulse response estimates obtained with m-sequences at different frame rates. The frame rate varied at 30, 40, 50, 60 and 70 Hz, and the results were averaged for 56 flies. Extraction of the heat content is an extension of hydrostatic dynamics in that warm features are elevated with respect to the mean conditions. The two-layer ocean model is based on the depth of the 20°C isotherm, which separates the lower- from the upper-oceanic layer.
Using historical hydrographic measurements, empirical relationships are determined from least square fits between the 20° and 26°C isotherm depths. Layer thicknesses relative to the depth of the 26°C isotherm derived from the SHA field when combined with mean temperature gradients provide crude synoptic estimates of the upper ocean's heat content. After the passage of Hurricane Opal, the upper-layer thickness changed by about 2ξ , which is consistent with the heat content decrease by 6QT (24 Kcal cm−2) along the hurricane track during the approximate time of encounter of 14 h. If this time rate of change is used, the upper ocean lost approximately 20 kW m−2. Given that the heat release from the ocean to the atmosphere based upon observations ranges between 10% and 15% , surface flux estimates presumably exceeded 2000 W m−2. Numerical simulations of the coupled response over the WCR were in excess of 2600 W m−2 (Hong et al. 2000), or roughly 13%–15% of the observed cooling signals derived from TOPEX altimetry.
We used a system identification approach to characterize the dynamics of optomotor responses in flies presented with simulated visual perturbations to their natural three-dimensional flight trajectories. Nevertheless, the response shapes go some way towards explaining two of the full impulse responses in Fig. When both visual fields moved forward and backward together, the stimulus was similar to thrust optic flow (though not perspective-corrected). The ipsilateral and contralateral impulse responses in this case have the same sign, and, therefore, when they are added, the result is noticeably similar to the h for thrust (Fig. 6C, black and gray traces). Likewise when each visual field moved differently, one forward and the other backward, the stimulus was yaw optic flow. In this case the impulse responses have opposite signs, and when one is therefore subtracted from the other, the result is similar to the h for yaw (Fig. 6C, black and gray traces).
Importantly, for this experiment, flies never viewed thrust or yaw stimuli , but rather saw uncorrelated motion to the left and right. Also, since the correlations for this stimulus were much smaller, the full responses (Fig. 6B,C) were scaled down for comparison. 6B has an inhibitory phase not found in the earlier thrust h estimate, and neither Fig. 6B nor C settles with the same time course as those estimated with coordinated stimuli.
This indicates the full responses are more than a linear combination of the independent ipsilateral and contralateral visual contributions to wing beat amplitude, and that nonadditive effects become important when left and right side motion are correlated. White noise analysis has been successful for both linear and nonlinear analyses in biological systems (De Boer and Kuyper, 1968;Marmarelis and Naka, 1972; Naka et al., 1979; Dickinson, 1990). It is often used to characterize responses of sensory neurons, and more rarely the behavioral output of a whole organism. But by comparison with visual processing in early motion circuits, much less is known about quantitative algorithms that transform the natural spatiotemporal patterns into the motor control of wing kinematics.
However, a fundamental model supposes that insects turn in the yaw plane to equalize motion between the eyes, in other words, the inter-ocular difference in optic flow is a feedback signal to stabilize yaw (Götz, 1975; Srinivasan and Zhang, 2000; Egelhaaf and Kern, 2002). A parameterless model allowed us to measure the precise temporal dynamics of these compensating responses, and resolve even subtle differences that varied with flow fields. Cross correlating the m-sequence with the wing beat responses produces an efficient estimate of the impulse response.
1D shows a partial m-sequence controlling yaw optic flow , the corresponding ΔWBA response , and the resulting estimate of the impulse response . We set this rate to a value known to produce robust optomotor responses in flies (Duistermars et al., 2007a). Although this parameter could influence the temporal resolution of the filter estimates, experiments indicated wide variation in frame rate had little effect (Fig. 1E, average of 56 flies). Whereas Bosart et al. document the favorable atmospheric conditions associated with trough interactions, the approach described herein focuses on the upper ocean's role in altering the tropical cyclone wind fields as Hurricane Opal passed directly over the WCR. Hydrographic measurements are combined with remotely sensed signals from TOPEX and AVHRR to provide synoptic assessments of upper-layer thicknesses and upper-ocean heat content relative to the 26°C isotherm in section 4. Concluding remarks concerning the application of these fields to Opal and the general approach to the intensity change problem are discussed in section 5.
A key finding here is that each flow field produced a unique impulse response in flies, and the estimated linear filters explained up to ~75% of the total response variance in subsequent experiments (Fig. 3). This of course does not imply that motion detection is linear, as it is known to require a nonlinear interaction , rather that motion responses are linear over a substantial range of operation. Furthermore, these results do not imply linearity within the entire cascade of sensory processing and motor control, but rather that the fully integrated behavioral system is well approximated by linear filters, as over half the variance is accounted for in most cases. Within some stages, the normal operating range of the fly might mitigate underlying nonlinearities.
For example, elementary motion detection shows strong nonlinearities over pattern velocity . However, recordings from wide-field integration neurons of the lobula plate have shown that the image speeds imposing the strongest nonlinearities generally occur above those encountered during flight . By modulating image motion with white noise at 50Hz such that each time step shifted the pattern in a random direction, we present multistep velocities from −188 to 188deg.s−1. It is important to keep in mind that pattern velocities outside the normal operating range of any element of the processing cascade will not evoke correlated behavioral steering responses and thus will not appear in the linear kernel estimates. The inset illustrates the experiment, in which random yaw patterns were modulated on the left and right with independent m-sequences.
The impulse responses for the ipsilateral and contralateral effect of optic flow on wing motion, estimated from 45 flies. The sum of the ipsilateral and contralateral kernels from A, the situation expected when both hemispheres move backwards and forwards together, roughly equivalent to the thrust stimulus in the previous experiments. The difference of the ipsilateral and contralateral kernels from A, expected when the fields move opposite and equivalent to the yaw stimulus in the other experiments. Another question of interest was whether different parts of the motion field affect the optomotor responses.
In other words, can we explain wing kinematics by the sum of responses to motion in different parts of the visual field? One difficulty is that as the size of the moving visual field is reduced, the optomotor response drops, so it becomes difficult to get a good estimate of h. Our approach consisted of treating the fly as a lumped control system, and measuring wing kinematics in response to visual perturbations, created by stimulating displacements through a cloud of points . The wing beat responses were then used to estimate the best linear filter (or impulse response, h) linking changes in the optic flow field to motor control of wing kinematics. We find that the dynamics of stabilization are different for different perturbations.
Additionally, despite great differences between individual measurement trials, the filters are highly predictive of the mean wing beat responses to novel random stimuli, usually accounting for half or more of the variance in stabilization responses. Several models account quite well for neuronal responses and flight behavior with feedback mechanisms (Collett and Land, 1975; Götz, 1975; Poggio and Reichardt, 1976; Dickson et al., 2008). It has been shown that models that account well for these responses are insensitive to wide variations in spatial textures, but instead are largely influenced by the dynamics of naturalistic optic flow (Lindemann et al., 2005). Additionally, studies in the blowfly have revealed that linear sums of the output from HS and VS neurons can encode translational and rotational components of self motion from naturalistic optic flow (Karmier et al., 2006).
However, the dynamical behavioral responses to optic flow and the linearity of these responses have not been studied in detail. Here we describe the temporal dynamics of optomotor responses of flies to perturbations in the optic flow field. As a reference to the optic flow, we describe motion relative to three perpendicular axes aligned with the fly's body (Fig. 1A). Translational motion along each axis is usually referred to as lift, thrust and slip; rotational motion around each axis is conventionally called yaw, roll and pitch. These axes are a basis for general translation and rotation, such that superimposing them can represent arbitrarily complex motion of the fly's body. The white noise analysis described here yields a precise linear dynamical profile of optomotor responses in Drosophila.
Coupled with genetic tools, white noise analysis could help determine the role that different neural circuits contribute to the response. Both the current methods and results can be applied in a reverse-genetic screen for the putative constituent neural circuits. For example, circuit breaking techniques such as reversibly inactivating genetically targeted neural microcircuits by way of the Gal4-UAS system will probably uncover novel visual pathways for decomposing the retinal flow field. Further characterization of nonlinear dynamical response components could make this approach stronger still.
It is important to note that the quantitative measures provided here, including the impulse responses and input–output variance functions, are not well understood at the neural circuit level. Therefore we have provided, in a genetic model organism, both a formalization of the operational algorithms and the basis by which to assay whether they are distorted. (A–D) Each h in column i is the mean of twenty measurements from a single fly, produced with randomly shifted m-sequences cross correlated with the fly's ΔWBA response.
Each response in column ii is the mean of twenty ΔWBA responses to aligned m-sequences, a black line flanked by one standard deviation in gray. The prediction of the linear dynamical model is superimposed on the response trace in red. Column iii plots the predicted against observed values from corresponding time points in ii, and shows the correlation coefficient above. Behavioral predictions using other impulse responses of other flies produce weaker correlations. Each column represents the responses of an individual fly, and the point shapes and colors represent behavioral predictions calculated with impulse responses measured from other individuals , plotted by their correlation value on the vertical axis. 3A–F reproduces the responsive linear filter for each type of optic flow (Fig. 3A–Fi).
The mean response to a novel 20 s m-sequence is shown in black (Fig. 3A–Fii), with one standard deviation in gray. The standard deviations of the ΣWBA responses are noticeably higher than the ΔWBA, a result of variations in focusing of wing shadows on the photodiodes. This affects the total amplitudes by shifting traces up and down, but does not reflect a less reliable metric.
The dynamical model predictions, overlaid in red, are normalized to the same maximum as the black traces for graphical comparison . Scatter plots show the model-predicted values against measured values at each time point (Fig. 3A–Fiii, sampled at 500 Hz) of the response traces. Above this are the correlation coefficients , the square of which yields the coefficient of determination, or the proportion of variability accounted for by the linear filters.
The correlation of all but one of the fits was above 0.7, so accounting for over half the explainable variability (the response to yaw in Fig. 3Dii was the weakest correlation and is discussed below). In the strongest case of response to sideslip, the filters accounted for over 75% (0.872) of the variability. Examining the scatter plots also allowed us to determine if static nonlinearities might explain even more of the variation–static because each predicted and measured pair were from the same time point, and nonlinear if we saw biases off a straight line that crossed the origin.
The plots (Fig. 3iii), however, show no systematic bias away from straight lines. The wing beat analyzer captures only a projection of three-dimensional wing stroke, the top down infrared shadow filtered through an optical mask, and thus could potentially sacrifice resolution, or introduce distorting effects, including nonlinearities such as saturation. But although wing kinematics are far more complex than the projections, there are several justifications for measuring the sum and difference of wing beat amplitudes.
First, we carefully focus the wing traces before experiments, confirming that flies can fixate a vertical bar in closed loop, and that the responses do not saturate during normal flight. Second, ΔWBA and ΣWBA have been shown to be proportional to yaw torque and axial thrust, respectively (Götz, 1987; Frye and Dickinson, 2004; Tammero et al., 2004). Third, lift and thrust are coupled during flight (Götz and Wandel, 1984), implying ΣWBA is relevant for both. Fourth, in response to a laterally expanding flow field, flies produce roll and yaw torques of similar magnitude in addition to pure sideslip axial force . The apparent cross-talk between sideslip and yaw is evidenced by a powerful active feedback loop between ΔWBA kinematics and visual sideslip (Tammero et al., 2004). Fifth and finally, we make the assumption that ΣWBA kinematics vary systematically with pitch torque, which is not unreasonable if the optomotor stabilization of nose-up pitch and a upward lift produces ΣWBA of similar sign.
Left and right wing stroke amplitudes and visual pattern position were digitized at 500 Hz and stored on a computer. However, the different responses may also suggest separate pathways for rotational and translational motion processing . First, optomotor response dynamics are strongly dependent on the perspective corrected optic-flow pattern. Second, approximate linearity extends to conditions where two types of motion field are superimposed, at least in the case of lift and thrust.
Such results could most easily be attained by decomposition of the motion field into basic components, then linearly combining the responses to generate a motor command – the very method by which we generated the prediction. This is important because optic flow fields contain an enormous amount of information for an animal in nature steering its course by vision but only sometimes orienting to the cardinal axes, as our experiments did. Yet by showing that two superimposed translational fields moving independently generate a response that is largely the sum of each shown individually, we demonstrate a simple method for how a fly might analyze and respond to an arbitrarily complex flow field. For the yaw stimulus, our pooled estimate of the impulse response accounted for only about 28% of response variance, a poor fit by linear filters. At first glance, this low value might be expected since during sensory independent active search, flies exhibit apparently spontaneous changes in yaw (Wolf and Heisenberg, 1990; Reynolds and Frye, 2007; Chow and Frye, 2008).
As repeatable as flight optomotor responses are, they show variation, and determining an underlying visual response requires averaging of multiple trials. Does this variation reflect purely random sensory-independent spontaneity or quantitative individual variation in the optomotor control system? Landing trajectories in honeybees show systematic inter-individual variation such that the angular velocity of the ground image is held constant upon approach, but the specific velocity value varies between individuals (Srinivasan et al., 1996).
