July A realistic locomotory model of Drosophila larva for behavioral simulations. Panagiotis Sakagiannis p.sakagiannis@uni-koeln.de 0 Anna-Maria Jürgensen 0 Martin Paul Nawrot 0 Computational Systems Neuroscience, Institute of Zoology, University of Cologne , Germany 2021 8 2021 1 15

The Drosophila larva is extensively used as model species in experiments where behavior is recorded via tracking equipment and evaluated via population-level metrics. Although larva locomotion neuromechanics have been studied in detail, no comprehensive model has been proposed for realistic simulations of foraging experiments directly comparable to tracked recordings. Here we present a virtual larva for simulating autonomous behavior, fitting empirical observations of spatial and temporal kinematics. We propose a trilayer behavior-based control architecture for larva foraging, allowing to accommodate increasingly complex behaviors. At the basic level, forward crawling and lateral bending are generated via coupled, interfering oscillatory processes under the control of an intermittency module, alternating between crawling bouts and pauses. Next, navigation in olfactory environments is achieved via active sensing and topdown modulation of bending dynamics by concentration changes. Finally, adaptation at the highest level entails associative learning. We could accurately reproduce behavioral experiments on autonomous free exploration, chemotaxis, and odor preference testing. Interindividual variability is preserved across virtual larva populations allowing for single animal and population studies. Our model is ideally suited to interface with neural circuit models of sensation, memory formation and retrieval, and spatial navigation.

Introduction

Building virtual agents that behave indistinguishably from living organisms is an endeavour at the interface of behavioral and cognitive neuroscience, artificial intelligence and behavior-based robotics. The pursued level of similarity to real animal behavior defines the target level of model abstraction, which can vary from low-level detailed neuroscientific models to high-level abstract cognitive architectures. Recent advances in tracking equipment have allowed highly resolved recordings of larva populations under diverse foraging conditions ( 1–4 ). Here, we ask how to build virtual larvae behaving indistinguishably from the real ones as captured by established larva tracking protocols. To this end we suggest a layered behavioral control architecture (Fig. 1). At the basic layer, we propose a locomotory model that features a bisegmental virtual larva body, coupled crawling and bending oscillators, and crawling intermittency. Integrating previously suggested features and novel data-driven hypothesis we compare diverse configurations against empirical data in order to choose the optimal locomotory model. Expanding our model to the upper layers of the control architecture enables increasingly more complex foraging-related behaviors (Fig. 1). In virtual larva populations with realistic inter-individual variability we evaluate model performance in simulations of several established behavioral paradigms.

Drosophila larvae possess a fairly tractable behavioral repertoire that is consistent across the 4-5 days of the larval life stages ( 5 ) and controlled by a conserved neural circuit structure throughout development ( 6 ), making it a formidable system for studying behavioral control and decisions ( 7 ). Most of the larval time is dedicated to foraging the environment for suitable nutrients while avoiding danger. Foraging consists of a combination of more basic behaviors: crawling, turning, digging into the substrate ( 8 ), and feeding, the latter even including cannibalizing conspecifics in extreme cases ( 9 ). In the absence of available food resources, larvae have to engage in free exploration to locate food patches ( 10 ). This behavior is intermittent, meaning it consists of bouts of activity interspersed by brief pauses generated via cessation and reinitiation of crawling ( 11 ), a property also reported for adult fly behavior ( 12, 13 ). Salient olfactory cues can trigger chemotaxis during which larvae employ active sensing to navigate along chemical gradients ( 14 ). Finally, novel odorants coupled to food reward induce olfactory learning enabling long-term behavioral adaptations ( 15–19 ). After reaching critical mass for pupation, homeostatic signals switch behavior towards food aversion, hypermobility and collaborative burrowing ( 20 ), terminating the feeding state and leading to pupation and metamorphosis.

Statistical regularities that govern foraging behavior have been unveiled by analysis both at the microscale of body kinematics and at the macroscale of larva trajectories ( 2, 21, 22 ). Crawling and turning have been in the main focus of recent studies ( 10, 23, 24 ) whereas tracking studies of feeding behavior remain scarce ( 25 ). Both, crawling and feeding behavior are indisputably of oscillatory nature ( 23, 26, 27 ) controlled by central pattern generating circuits. With respect to turning, it is still debated whether individual turns should be considered as discrete reorientation events that are temporally non-overlapping with crawling bouts ( 10 ), or whether turning occurs in an oscillatory fashion generating turns both during crawling and during pauses ( 24, 28 ). The latter is supported by detailed eigenshape analysis confirming that larvae rarely crawl straight, rather forward locomotion is always accompanied by continuous small amplitude lateral bending ( 29 ). It follows that crawling does not exclude bending rather the two strictly co-occur. In contrast, both feeding and crawling movements require mouth-hook motion recruiting the same effector system, thus they can be considered competing, mutually exclusive behaviors ( 7, 30 ). Finally, it is unclear whether bending and feeding can overlap.

Modulation of exploratory behavior under salient olfactory

Simulations were run via the Larvaworld behavioral analysis and simulation platform,

available at https://github.com/nawrotlab/larvaworld input during positive and negative chemotaxis has been extensively studied ( 14, 16, 31, 32 ). There is general consensus that during appetitive chemotaxis increasing odor concentrations suppress pauses and turns leading to longer lasting bouts of activity, while turns are promoted during down gradient navigation. Turning is biased by the detection of minor concentration changes during lateral bending, a process described as active sensing ( 24, 28, 33 ). The opposite effect has been reported during aversive chemotactic behavior whether due to punishment or reward omission ( 34 ). Larva behavioral preference under conflicting olfactory stimuli has been established as a population-level metric in multiple settings from quantifying the formation of memory after associative learning ( 15–17, 35, 36 ) to detecting individual differences in genetically identical larva strains ( 37 ). This sensory-driven behavioral modulation does not seem to be affected by social cues, justifying the study of individual larva kinematics even during population-level experiments ( 38 ).

Results

Kinematic analysis of larva locomotion. We start out with the kinematic analysis of experimental larva trajectories and body postures in order to infer and parametrize several aspects of larva locomotion that will inform our modeling approach. Using diverse metrics that capture spatial and temporal dynamics we specifically assess the oscillation of forward velocity during individual peristaltic strides, the influence of this oscillation on lateral bending, the intermittent nature of crawling, and the inter-individual variability of a number of locomotion-related parameters across different larvae.

Color annotation of larva trajectories in Fig. 2 A illustrates nicely that forward locomotion consists of consecutive steps (strides) that are characterized by an alternating increase/decrease of the locomotion velocity vˆ . To characterize this oscillation we detected all strides performed by an individual animal and verified their stereotypical structure in terms of strude duration, resulting body displacement, and the phase-dependence of vˆ (Fig. 2 D). Scaling both, displacement dstr and velocity to the individual larval body-length increases stride stereotypicality independent of larva size. This analysis justifies an oscillatory model process (crawler, ) that generates forward-velocity in subsequent stride cycles (crawler, Fig. 1 B) is introduced for model comparison.

Our analysis of orientation velocity θˆ˙or during crawling strides reveals that it is phasically coupled to the stride phase exhibiting an increase around 32π of the oscillatory cycle (Fig. 2 H). This implies phasic interference of the crawling and the lateral bending mechanism (Fig. 1 B), which is incorporated in our model (see Material and Methods). A plausible mechanistic explanation featuring bodily interference of crawling and bending is suggested in the discussion.

Larvae transiently pause crawling before re-assuming it resulting in sequences of concatenated strides (stridechains) intermitted by brief crawl-pauses (Fig.2 C). We analyzed the distributions of stridechain length and pause duration in the experimental dataset. The limited duration of the recordings (3’) does not allow assessment of individual differences therefore the stride chain and pause bouts over a population of 200 larvae have been pooled together. Testing power-law, exponential and log-normal distributions revealed the highest quality of fit for the log-normal distribution for both parameters (see 2 | A

B

Material and Methods). In our model we propose a behavioral intermittency module (Fig.1 B) that samples from the empirically fitted model distributions and controls cessation and re-initiation of crawling (see Material and Methods).

Behavioral architecture. Larval behavior is hierarchically structured in the sense that simple behavioral motifs such as crawling, bending and feeding motions can be integrated into more complex behavioral modes such as exploration, taxis and exploitation. It has been proposed that animal behavioral hierarchy is reflected by the underlying neuroanatomy as a hierarchy of nested sensorimotor loops ( 39 ). A functional modeling paradigm that exploits this idea regards the neural system 1)1.0 ce0.8 s ( ity0.6 c loe0.4 v led0.2 a c s0.0130

150 time (sec) 135 140 145 155 160 165 170 135 140 145 155 160

165 150 time (sec) 400 200 ) c e s / g e d ( y it c o l 200 rve a l 400 ung

a 0 170 Fig. 2. Kinematic analysis of the Drosophila larva in locomotion. A Individual larva trajectory tracking a rear point along the midline of the animal. Trajectory color denotes the forward velocity vˆ from 0 (red) to maximum (green). Inset focuses on the track slice analyzed in C and G. Dark green rectangle denotes a single stride described in B. B: Sketch of a single crawling stride. The larva first stretches its head forward, anchors it to the substrate and then drags its body forward via peristaltic contraction. Scaled stride displacement dstr is defined as the resulting displacement d scaled by the body-length l. C: Forward velocity vˆ during the 40 s track slice selected in A, inset. Green and red markers denote the local minima and maxima used for stride annotation. Individual strides are circumscribed by vertical dashed lines. Successive strides constitute uninterrupted stridechains (white). Epochs that do not show any strides are annotated as crawl-pauses (gray). Velocity is scaled to the larva body-length. D: Forward velocity of head, midpoint and tail as a function of the stride cycle. All detected strides of the tracked larva have been interpolated to a stride oscillation cycle of period 2π. Solid lines denote the median, shaded areas the lower and upper quartiles. Velocity is scaled to the larva body-length. E: Same trajectory as in (A) now tracking the head segment. Color ˙ denotes the orientation angular velocity θˆor from 0 (red) to maximum (green). F: Definition of bending angle θb and orientation angle θor for the original 12-segment (blue) ˙ and the simplified 2-segment (red) larvae. G: Three angular parameters during the same track slice shown in (C). Bending angle θˆb, bend and orientation angular velocities θˆb, ˙ θor are shown. Background shadings denote left and right turning bouts. For illustration purposes only turns resulting in a change of orientation angle Δθˆor > 20◦ are shown. ˆ H: Same as D but showing the average absolute orientation angular velocity θˆor during the stride cycle. as a layered control (subsumption) architecture ( 40 ) where low-level stereotyped reflexive and repetitive behaviors are autonomously generated by localized peripheral motor circuitry while multisynaptic loops involving more centralized neural circuits only act as top-down modulators on the local circuits in order to coordinate global and complex behavioral control.

The central idea is that energy-efficient decentralized neural control is the rule, while higher centers are recruited only when more extensive integration is needed. Furthermore there are only limited degrees of freedom by which higher layers can influence local sensorimotor loops e.g. by starting/stopping or accelerating/decelerating their autonomous function (see Discussion). Layered control architectures have been used mainly

Sakagiannis et al.

in behavior-based robotics allowing sequential calibration and modular integration of partial neuroscientific models under a common framework ( 39, 40 ).

We here propose a trilayer behavioral control architecture for Drosophila larva foraging as illustrated in Fig. 1 A. The bottom layer comprises three basic stereotyped behaviors: crawling, turning and feeding. Each is realized by a low-level local sensorimotor loop between motor effectors and sensory feedback - mainly proprioception and mechanoreception - and additionally gustatory input in the case of feeding. Integration of these basic behaviors gives rise to composite behaviors. Exploration in stimulus-free conditions requires crawling and turning while integration of all three basic elements allows exploitation of a nutritious substrate. The intermediate layer introduces salient sensory stimulation of different modalities therefore allows reactive behavior in the face of presented risks and opportunities. Modulation of exploratory behavior under appetitive or aversive stimulation enables coherent navigation along sensory gradients. Here, we consider odor-driven chemotaxis as a process of active sensing in which the larva constantly samples the odor concentration during lateral bending motions enabling odorscape navigation and odor source discovery. At the top layer, associative learning during prior experience biases the evaluation of recurring sensory stimuli resulting in modulation of innate odor valence and providing experience-dependent adaptation of navigation.

The proposed behavioral architecture is naive to the underlying neural mechanisms that generate the respective behaviors. We hereby populate the architecture with diverse configurations of candidate mechanisms and explore autonomous and integrated control exerted by each subsequent layer in increasingly-complex behavioral simulations from exploration to chemotaxis to adaptive learning in odor preference experiments.

Locomotory model. We simplify the larva to a two-segment body (Fig. 2 F). This abstraction allows describing the body state at any point in time through only three parameters. These are (i) position of a selected midpoint (Fig. 9), (ii) absolute orientation of the front segment θor, and (iii) bending angle θb between the front and rear segments. This approach is in line with the common practice of quantifying larva bending via a single angle between one anterior and one posterior vector, although the chosen vectors vary between study groups ( 14, 34, 41 ). Body dynamics as analyzed in the empirical data depends on instantaneous linear (Fig. 2 A-C) and angular velocities (Fig. 2 E-G) generated through crawling and bending respectively. Kinematic analysis on the locomotion of these simplified real larvae allows us to realistically calibrate bisegmental virtual larvae resembling the real ones in multiple spatial and temporal parameters (see Materials and Methods). For a demonstration of the larva-body bisegmental simplification see Video 1. The individual trajectory depicted in Fig. 2 can be seen at its full length in Video 3, while the slice depicted in the inset is shown in Video 4.

Locomotion of the bisegmental body is achieved via dynamic coupling of forward crawling and lateral body bending. Crawling moves the midpoint along the orientation vector. Bending reorients the front segment by rotation around the midpoint. Forward displacement restores θb back to 0 by gradually aligning the rear segment to the orientation axis.

We adopt the hypothesis that attributes alternating leftright bending to an underlying oscillatory neural process and compare the lateral oscillator model described by Wystrach et al. ( 24 ) to a simple sinusoidal oscillator. This model (turner) applies torque oscillations on the two-segment body, which acts as a restorative torsional spring (see Materials and Methods). The turner can be coupled to a second oscillator producing crawling strides (crawler) in the form of linear velocity oscillations as suggested by our kinematic analysis. Under the constraints of the subsumption architecture paradigm these effectors can be influenced by higher-order circuits only in a limited number of ways. Frequency modulation and stride initiation/cessation are the only possible top-down modulations on the crawler. Likewise, the turner can receive olfactory input 4 | affecting both the frequency and the amplitude of oscillation.

To quantify the contribution of each of the locomotory model’s components to replicating real larva kinematics we perform a broad model comparison study across diverse model configurations (Fig. 3). The behavioral metrics used for evaluation are structured in five categories covering angular and forward motion, reorientation, spatial dispersion and stridecycle structure. Overall, the bisegmental is superior to the single-segment body in capturing angular motion metrics, predominantly due to the decoupling of the bending velocity θ˙b from the orientation velocity θ˙or (in the single segment body these are considered identical). A realistic crawling oscillation allows much more accurate assessment of stride-cycle dependent metrics compared to non-oscillatory constant-speed forward motion (despite the lack of a velocity oscillation in the latter case, strides might still be detected due to noise). Finally, both oscillator coupling and crawl-intermittency contribute to better fitting of the empirical data. Our final model is represented by the far bottom right column, exhibiting the smallest error.

The detailed structure and optimal configuration for the basic layer of the behavioral architecture as suggested by model comparison is illustrated in Fig. 1 B, specifying the interplay between the oscillators. Two additional features are implemented based on our empirical analyses. First, the crawler and turner are phasically coupled such that turning is suppressed during a defined phase interval of the stride cycle, reflecting our finding in Fig. 2 H. Second, crawler-generated chains of concatenated strides (stridechains) are intermitted by brief pauses during which the crawler-induced interference is lifted, resulting in more acute turning events. The duration distribution of stridechains and pauses are fitted to the empirical data. The pipeline for model calibration is described in detail in Material and Methods. For a demonstration of the locomotory model in different configurations see Video 2. We note that in the current implementation effectors receive no sensory input from the environment. Sensory feedback is introduced only at the intermediate reactive layer (Fig. 1 A) via an olfactory sensor located at the front end of the body (head). All model parameters are shown in Table 1.

Inter-individual differences are crucial for achieving realistic population-level behavior. To capture variability across larva we computed three crawl-related parameters, body-length l, crawling frequency fc, and scaled stride displacement dstr across a population of 200 experimental larvae. The univariate and bivariate empirical distributions are illustrated in blue in Fig. 4. When generating virtual larva populations for the behavioral simulations we sample all three parameters from a multivariate Gaussian distribution fitted to the empirical data. This preserves the linear correlation structure. A generated set of the three parameters completely defines the crawling oscillation of an individual virtual larva. The bivariate projections are shown in red in Fig. 4 with parameters given in Table 1 in blue.

Simulation of behavioral experiments. The layered behavioral architecture in Fig.1 is exploited in modeling as it justifies sequential calibration and evaluation of subsequent behavioral layers from the bottom to the top, as for any specific behavior to be successfully realized, only integration up to a certain layer is required. In this section we simulate increasingly more complex behavioral experiments using virtual larva popula

Single-segment body

Two-segment body Uncoupled

Coupled

Uncoupled Intermittent Coupled Intermittent Oscillator coupling & Intermittency tions. Starting from stimulus-free exploration we advance to chemotactic navigation and finally to adaptive odor preference experiments. Individual virtual larvae behave independently of each other as they move through the spatial arena and odorscape ( 38 ).

Free exploration. Larvae explore a stimulus-free environment, dispersing in space from their starting positions. As there is no food nor any salient odor gradient, integration of crawling and bending behaviors is sufficient. We compared free exploration in populations of 200 virtual and real larvae respectively (Video 6). Statistical evaluation showed a good agreement of simulated and empirical data with respect to spatial dispersion of larvae from their initial position (Fig. 5 A,B), total distance traveled, time fraction allocated to crawling and number of performed strides (Fig. 5 C). For a demonstration of virtual and real larvae exploring a dish see Video 5 while for a comparative assessment of their dispersion dynamics see Video 6.

Chemotaxis. Chemotaxis describes the process of exploiting an odor gradient in space to locate an attractive or avoid a repelling odor source. An olfactory sensor (olfactor) placed at the front end of the virtual body enables active sensing during body bending and allows detection of concentration changes that modulate turning behavior accordingly (see Methods). To assess chemotactic behavior in our model we reconstruct the arena and odor landscape (odorscape, see Methods) of two behavioral experiments described in ( 14 ). In the first, larvae

Sakagiannis et al.

are placed on the left side of the arena facing to the right. An appetitive odor source is placed on the right side. The virtual larvae navigate up the odor gradient approaching the source (Fig. 6 A), reproducing the experimental observation in Fig. 1 C in ( 14 ). In the second, both the odor source and the virtual larvae are placed at the center of the arena. The larvae perform localized exploration, generating trajectories across and around the odor source. (Fig. 6 B), again replicating the observation in Fig.1D in ( 14 ). Fig. 6 E and F show the average time-varying odor concentration encountered by the virtual larvae along their trajectories, replicating the estimations from real larval tracks in ( 14 ). Two sample simulations can be seen in Video 7.

Odor preference test. We simulate the odor preference paradigm as described in the Maggot Learning Manual ( 42 ). Larvae are placed at the center of a dish containing two odor sources in opposite sides and left to freely explore. The odor concentrations are Gaussian-shaped and overlapping, resulting in an odorscape of positive and/or negative opposing gradients. After 3 minutes the final situation is evaluated. The established population-level metric used is the olfactory preference index (PI), computed for the left odor as P Il = Nl−NNr where Nl and Nr is the number of larvae on the left and right side of the dish while N is the total number of larvae.

The extend of olfactory modulation on the turning behavior is determined by the odor-specific gain G (see Materials and Methods). As this is measured in arbitrary units, we first need to define a realistic value range that correlates with )4.8 m m l( th4.0 g n e ly d ob3.2 2.4 ) z (Hc2.0 f y c n euq1.6 e lfr w ra1.2 c )0.30 (trs pd0.24 e ts e ird0.18 ts 0.12 3.2 4.0 4.8 body-length l(mm) 1.2 1.6 2.0 crawl frequency fc(Hz) 2.4 0.12 0.18 0.24 stride step dstr(0.)30 Fig. 4. Parameters of individuality: empirical and fitted distributions. Diagonal: Histogram and kernel density estimates (KDE) for body-length l, crawling frequency fc and mean scaled displacement per stride dstr across a population of 200 larvae in the experimental dataset. Below: Bivariate projection of 3-dim. KDE outlined contours for each parameter pair. Above: Red ellipses represent the bivariate projections of the 3-dim. fitted Gaussian distributions at 0.5, 1, 2 and 3 standard deviations. In our model this Gaussian is used to sample a parameter set for each individual larva. The blue dots denote the empirically measured parameters. the behaviorally measured PI. We perform a parameter-space search independently varying the gain for left and right odors and measuring the resulting PI in simulations of 30 larvae. The results for a total of 252 gain combinations within a suitable range of G ∈ [−100, 100] are illustrated in Fig. 7 A. Simulation examples for one appettitive and one aversive odor are shown in Video 8.

In order to simulate larval group behavior in response to an associative learning paradigm we interface our behavioral simulation with the spiking mushroom body (MB) model introduced in ( 43 ) (Fig. 7 C). It implements a biologically realistic neural network model of the olfactory pathway according to detailed anatomical data using leaky integrate-and-fire neurons ( 44 ). The MB network undergoes associative plasticity at the synapses between the Kenyon cells and two MB output neurons as a result of concurrent stimulation with an odor and a reward signal. Both, odor and reward is simulated as spike train input to the receptor neurons and the reinforcement signalling dopaminergic neuron, respectively. The model employs two output neurons (MB+, MB-), representing a larger number of MB compartments associated with approach/avoidance learning respectively ( 45 ). The initially balanced firing rates between MB+ and MB- are skewed after learning and encode the acquired odor valence ( 46, 47 ) here defined as M Bout =

M B+ − M B− M B+ + M B− ∈ [−1, 1].

We first trained the MB model via a classical conditioning experiment where, in each conditioning trial, it experiences an odor (conditioned stimulus, CS+) in combination with a 6 | sugar reward during 5 min, following the standard training protocol in ( 42 ). 5 groups of 30 MB models undergo between 1 and 5 sequential conditioning trials ( 48 ). The resulting odor valence MBout from each MB model was converted to an odor gain G via a simple linear transformation and used to generate a virtual larva (Fig. 7 B). Each population of 30 larvae was then tested in an odor preference simulation. The larvae were placed on a dish in presence of the previously rewarded odor (CS+) and a neutral odor in opposite sides of the dish (Video 8), again following standard experimental procedures ( 42 ). To obtain robust results we replicated the experiment 100 times per population with a different random seed for a total of 600 simulations. The obtained preference indexes (PI) are illustrated in Fig. 7 D. The PI increase with increasing number of trials as well as its saturation resembles empirical observations ( 48 ). Note that the variability of the PI across the 100 simulations per condition is introduced solely by the behavioral simulation and resembles that seen across real experiments.

The current implementation only sequentially couples a trained MB model to be tested in a behavioral simulation. In the discussion we further elaborate on a possible extension featuring their closed-loop integration allowing for full behavioral simulations of both the training and the testing phase of the associative learning paradigm in a virtual environment.

Discussion

Neural pathways. The neural mechanisms enabling most of the basic behaviors have been extensively studied. Crawling occurs via fairly stereotypical repetitive strides. Head and tail segments initiate the stride concurrently with a ’visceral piston’ mechanism followed by a laterally symmetric peristaltic wave traversing neighboring segments longitudinally from back to front ( 23 ) (Fig. 2 B). Segmental central pattern generators (CPGs) coupled via premotor-involving intersegmental short- and long-range connectivity motifs constitute the underlying neural circuitry ( 26, 49, 50 ). Lateral bending results from asymmetric contraction of body musculature initiated at the thoracic segments ( 51 ). Finally, feeding is generated via a network of mono- and multi-synaptic sensorimotor loops from enteric, pharyngeal and external sensory organs to motor neurons controlling mouth-hook movement, head-tilt and pharyngeal pumping ( 27 ).

All three basic behaviors are autonomously generated by local circuitry while higher brain centers modulate their activity via descending input, mediated by dopamine, serotonine, achetylocholine, octopamine and other neurotransmitters ( 27, 52–56 ). It has been suggested that the transition between exploration and exploitation (feeding) is acutely induced via dopaminergic signaling ( 57 ) while their long-term balance is regulated via hugin-mediated homeostatic neuromodulation ( 58 ). Identification of sensory pathways towards motor effector neuropiles further elucidates the role of interoception in behavioral modulation ( 59 ).

The neural mechanisms that underlie olfactory modulation of the basic locomotory behavior are also under intense investigation. Chemotactic approach and avoidance to innately valenced odors has been attributed to a predominantly innate pathway involving the antennal lobe (AL) and its direct projection to the lateral horn (LH), both in the larva ( 56 ) and in the adult fly ( 60–62 ). Modulation of learned odors strongly involves top-down control by the MB in juvenile ( 45, 55 ) and adult ( 31 ) stages. Both pathways have similar modulating effects on foraging behavior and are likely integrated in a

Sakagiannis et al.

B D premotor network downstream of the AL ( 16, 63 ). In the sensorimotor loop, descending pathways involving the LH control cessation of crawling, possibly triggering sharper reorientation when navigating down-gradient, facilitating chemotaxis ( 64 ). Finally, the internal homeostatic state (e.g. starvation vs. satiation) regulates behavior via neuromodulatory transmitter release at multiple levels, including AL, LH and MB ( 56 ). Existing computational models. Crawling and bending mechanisms have been successfully captured in previous computational models. CPG models of segmentally repeated paired excitatory and inhibitory (EI) neuronal rate units, standing for average EI population activity, can autonomously generate forward and backward crawling, possibly involving proprioceptive feedback ( 65, 66 ) and the contribution of the visceral-piston mechanism to the peristaltic cycle has been assessed in a biomechanical model ( 67 ). The hereby adopted, continuous oscillatory lateral bending process has been implemented as a pair of bilateral mutually inhibitory EI circuits ( 24 ). The idea has been elaborated in a neuromuscular model generating autonomous forward / backward crawling and turning as well as their interplay during free exploration in a 12-segment larva body by modeling segmental localized reflexes and substrate frictional forces and assuming empirically informed axial and transverse oscillatory frequencies ( 68 ). Chemotaxis has also been modeled computationally either in stochastic transition models assuming discrete behavioral states of crawling (runs) and turning (head-casts) ( 69 ) or by introducing olfactory modulatory input on the underlying locomotory circuit ( 24, 68 ).

We propose here that modeling locomotion as coupled intermittent oscillatory processes (Fig. 1 B) is adequate for generating realistic larva kinematics as these are captured via larva tracking. This locomotory model efficiently summarizes the underlying CPG activities into global linear and angular velocity oscillations ( 23, 26 ) and accurately reproduced a number of experimental observations. It is therefore well suited for the bottom layer of the proposed behavioral architecture (Fig. 1). The latter though allows for a modular approach, where the level of abstraction can be chosen independently for each individual module as exemplified by combining this locomotory model with a spiking neural network that captures plasticity in a central brain neuropile (MB) at the top layer. It follows that the proposed control architecture affords any substitution of the currently selected oscillator modules by one of the aforementioned more-detailed neuromechanical models if a higher degree of biological realism is pursued. Crawl-bend interference. Crawling includes mouth hook motion. Specifically, the first phase of a crawling stride consists of concurrent forward motion of head and tail segments, aided by a ’visceral pistoning’ mechanism that generates forward displacement of the gut. Subsequently, the mouth hooks anchor the head to the substrate so that the second phase of peristaltic motion can drag all other segments forward as well, completing the stride ( 23 ). Crawling and bending partially recruit the same effector neural circuitry and body musculature at least at the level of the thorax. Peristaltic motion during crawling includes sequential symmetric bilateral contraction of all segments while bending occurs due to asymmetric unilateral contraction of the thoracic segments. This partial effector overlap could result in interference between the two processes. Indeed here we report an increase of orientation velocity during a specific phase of the stride cycle (Fig. 2 H), synchronous to an increase of head forward velocity (Fig. 2 D). The latter coincides with the stride phase when the head is not anchored to the substrate and therefore free to move laterally. When applying attenuation of lateral bending outside a phase interval [ π2 , π] of the stride cycle we managed to accurately reproduce the empirical relation (Fig. 8 A).

A reasonable hypothesis would then be that the asymmetric thoracic contraction generating lateral bending is only possible while the head is not anchored to the substrate therefore during a specific phase interval of the stride cycle. We suggest that crawling phasically interferes with lateral bending because of these bodily constraints. A consequence of the proposed hypothesis is that the amplitude of turns generated during crawl-pauses is larger in comparison to those generated during crawling because during pauses the crawling interference to lateral bending is lifted. We postulate that it is exactly this phenomenon that dominates the description of larva exploration as a Levy-walk with non-overlapping straight runs and reorientation events, where the minor orientation changes taking place during crawling are neglected ( 10, 70 ). We note that in the implemented model the turner neural oscillation is not inhibited at all during crawling strides as we consider this merely a bodily interference, although the resulting torque might eventually not be applied to the body depending on the stride-cycle phase.

Behavioral intermittency. Larval locomotion is intermittent meaning that crawling runs are transiently intermitted by brief pauses. The spatial dynamics of these alternating states have been studied in the context of motion ecology. During free exploration, power-law distributed runs, in line with Levywalk theoretical models ( 10, 70 ) and diffusion-like kinematics have been reported ( 32 ) while the speed-curvature power-law relationship has been disputed ( 71–74 ). Regarding the temporal dynamics of intermittency, the duration distributions of activity and inactivity bouts captured via larva-tracking recordings have been studied. More specifically, the duration of inactivity bouts has been reported to follow a power-law while that of activity bouts a log-normal distribution ( 11 ), partly in line with findings in adult-fly studies ( 12, 13 ). In all these studies micro-movements like feeding and lateral bending could not be detected due to technical constraints. Thus, the reported inactivity and activity bouts can be regarded as crawl-pauses and crawl-runs respectively, the duration of the latter being equivalent to our discretized stridechain-length metric.

Our analysis reveals that both distributions are approximated best by log-normal distributions (Fig.8,B). The lognormal stridechain distribution is in line with previous findings, while the log-normal pause distribution diverges from a previously reported power-law ( 11 ). This might be attributed to the short, 3-minute duration of the recordings in the present study contrary to the long, up to one hour recordings used in the latter.

Computational models of behavioral intermittency are scarce. A recent study presented a simple binary-neuron model exhibiting state transitions between power-law and non A

-100 -100 GR -50 , n i a g r 0 o d o t h igR 50 100 power-law regimes via self-limiting neuronal avalanches and proposed a plausible underlying mechanism that explains initiation/cessation of crawling ( 11 ). In the current model we remain agnostic to the intermittency generative process and simply sample pause duration and stidechain length from the empirically fitted distributions. Further neuroanatomical evidence of the underlying circuits is needed to elucidate the neural mechanism of intermittent crawling.

Architecture extensions. The here adopted subsumption architecture paradigm is mainly used in behavior-based robotics ( 75 ) and is supported as a theoretical framework for the description of the nervous system of living organisms ( 39 ). According to this, neural control of behavior consists of nested sensorimotor loops where more stereotyped reflexive behaviors are autonomously generated by localized neural circuitry at a faster timescale without recruiting more centralized resources. Once there is need for more extensive integration, e.g. in order to react suitably to unexpected sensory stimulation, slower multisynaptic loops enable higher centers to modulate local circuits achieving more coordinated global behavioral control. In other words, the nervous system is considered a multilayered control architecture within which higher layers subsume their subordinate layers into comprehensive behavioral modes. The main idea governing this paradigm is that there are only limited degrees of freedom in which higher layers can influence the lower ones e.g. by initiation/cessation or acceleration/deceleration of their autonomous function.

The modularity of the proposed architecture facilitates further extensions capturing novel aspects of the body, the nervous system, the metabolism or the environment as shown in Fig.1, allowing for more refined or more complex behavioral patterns. We here summarize some plausible extensions. Feeding. Despite their central role in foraging, feeding mechanisms have not been modeled computationally nor has their integration with the exploratory circuitry in the context of substrate exploitation. Feeding behavior also consists of repetitive stereotypical movements of the head, mouth hook and ingestive muscles ( 27 ). Every cycle is an intertwined sensorimotor loop under instantaneous feedback from the environment and slower regulation by higher self-regulatory centers. In adult flies successive feeding movements are organized in intermittent feeding bouts, interspersed by locomotory or idle periods ( 76 ). Although detection of larval feeding motions in freeforaging conditions is technically difficult due to their small amplitude, it is reasonable to assume these are also structured in intermittent bouts. The frequency of this repetitive motion has been reported to vary at least from 1 to 2.5 Hz. Therefore feeding behavior can be implemented as a third oscillator (feeder). Top-down modulation affecting initiation/cessation and oscillation frequency can be assumed, similar to crawling while sensory feedback can be implemented as recurrent modulation depending on successful ingestion.

Considering oscillator coupling, crawling and feeding cycles partially compete for control of the same effectors as they both recruit the head and mouth muscles. It has been reported that the number of mouth hook motions over a given duration of foraging does not differ between rover and sitter larva phenotypes and is not correlated to the amount of food ingested, although rovers crawl more and feed less than sitters ( 30 ). Therefore an individual mouth hook motion can equally Sakagiannis et al. be part of either a crawling or a feeding cycle. The conclusion drawn is that individual crawling and feeding cycles are competing mutually exclusive behavioral motifs meaning crawling and feeding bouts can alternate but do not overlap. Conversely there is no empirical evidence on the potential coupling of feeding and bending motions. Integration of the feeder in the current locomotory model is illustrated in Fig.1 B. Multimodal sensory feedback. Sensory feedback from the environment can be extended to other modalities beyond olfaction. Mechanosensation can be implemented via additional touch sensors around the body contour. This will allow detection and behavioral modulation by conspecific contact (collisions) ( 77 ) and external mechanosensory stimulation driving startle/evasion (hunch/bend) ( 78 ) or navigation along wind gradients (anemotaxis) ( 79 ). Accordingly, temperature and light sensors could allow thermotaxis ( 80–82 ) and phototaxis ( 83 ). The respective environmental sensory gradients can be Gaussian as in the case of odorscapes or linear along a certain arena axis. Behavioral modulation can be either summed up across modalities or separately applied to the bending, crawling and feeding effectors. Integration of additional sensory modalities is illustrated in Fig.1 A.

Olfactory learning in closed loop behavioral simulations. We have shown an open-loop simulation of the classical conditioning paradigm (Fig. 7 B-D) reproducing a basic experimental result in the fruit fly larva ( 36 ). This modeling approach can be extended in multiple ways. First, the larva demonstrates a number of interesting learning abilities/features that require synaptic and circuit mechanisms such as differential conditioning ( 15, 16, 36 ), extinction learning ( 84 ), and relief learning ( 35, 45, 48, 85 ). Interfacing neural network simulations of these mechanisms with our behavioral model allows to directly compare virtual and empirical behavioral experiments, both for the typical group assays and for individual animals. Second, while information about odor concentration is provided via olfactory sensing, direct input from a feeder module could provide the reward stimulus that activates the dopaminergic pahtway required for synaptic plasticity the mushroom body ( 45, 86 ). This would further allow realistic foraging scenarios with food depletion and competition. Closing the loop from active sensing to associative memory formation and behavioral control requires to synchronize a (spiking) neural network at the adaptive layer with the sensory (reactive layer) and locomotor modules (basic layer). This will enable the simulation of virtual larva experiencing spatial and temporal dynamics in a virtual environment or on a robotic platform ( 87 ). Such model approaches will allow to test model hypotheses on sensorymotor integration and to infer predictions for experimental interventions such as optogenetic stimulation ( 45 ) or genetic manipulations ( 88–91 ).

Materials and Methods

Dataset description. The larva-tracking dataset was obtained by M.Schleyer and J. Thoener at the Leipzig Institute of Neurobiology (https://doi.org/10.12751/g-node.5e1ifd). It consists of 200 third-instar larvae tracked at a framerate of 16 Hz for 3 minutes while exploring a non-nutritious substrate. 12 points are detected along the longitudinal axis of each larva. Detected collisions have been excluded and the data has been filtered with a first-order butterworth low-pass filter with a cutoff frequency of 2 Hz in order to decrease tracking-related noise but retain the behaviorally relevant crawling frequency of ≈ 1.5 Hz. The effect of inadequate and excessive filtering is illustrated in Video 9.

All data processing and all simulations were performed using the Larvaworld behavioral analysis and simulation platform programmed in python. We made Larvaworld freely available at https://github.com/nawrotlab/larvaworld. In Larvaworld, simulated and empirical data are treated indistinguishably, meaning that the exact same analysis pipeline and behavioral metrics are applied in both.

Crawler oscillator. Crawling behavior is modeled as an oscillatory process. Each oscillation generates a cycle of forward velocity v increase-decrease resulting in displacement of the larva along the axis of its front-segment, simplistically modeling the result of exactly one peristaltic stride. An analytically tractable curve is fitted to the average empirical velocity curve measured during strides (Fig. 2 D): v∗ = dstr · fc · (0.6 cos (Φ − Φmax) + 1) [1] where dstr is the displacement per stride, scaled to the larval body-length l, fc being the crawling frequency and Φ the instantaneous phase of the oscillation iterating from 0 to 2π during an oscillatory cycle. Φmax = π is the phase where the maximum velocity occurs. The equation ensures a constant vds=tr Nde(svp∗i,tneCfc· vc∗h)a.nnCgeiss. selected so that the variance across

Gaussian noise is applied so that strides fits the empirical observations (Fig. 8 A,top). The 3 parameters (l,dstr and fc) defining the velocity curve are sampled from empirical data, retaining their paired correlations (Fig. 4). The oscillatory process can only be halted/initiated at the start of a cycle meaning that once a stride is initiated it will be completed.

To calibrate Φmax in the stride cycle, a sliding window analysis was performed (data not shown). dstr was computed for all possible Φmax in windows of the reference stride duration 10 | fc−1, selecting the one displaying the minimum variance for displacement, which results in more stereotypical strides. Turner oscillator. The lateral oscillator model described in ( 24 ) assumes an underlying oscillatory process driving alternating bending to the left and right side. The oscillator consists of two mutually inhibiting components (L vs R) that quickly settle in antiphase, while adaptation ensures that periodic transitions occur. The system is driven by external activation At. The baseline activation At = 20 was held constant as in the original implementation ( 24 ), resulting in an average 0.3 Hz oscillation frequency. Perturbations of this external drive cause transient changes in both amplitude and frequency, up to transient loss of oscillation. This feature is exploited during olfactory modulation (see Olfactory sensor). Turner activity is the instantaneous difference in the firing rates Δf = (fL − fR) and is scaled by a coefficient ct to generate the oscillating torque T = ct · Δf . This is applied to the body which is modeled as a torsional spring of restorative spring constant k, causing lateral bending θb. The angular velocity is attenuated by angular damping ratio z. It holds that : θor = T − z · θ˙or − k · θb ¨

Gaussian noise i∗s applied to both the input activation At = N At∗, nA · At and the output turner activity Δf = N Δf ∗, nT · Δf ∗ .

In its original implementation the model deliberately neglects two aspects of the real turning behavior of the larva. First there is no distinction between bending θ˙b and orientation θ˙or angular velocities. Second there is no correction of the bending angle θb due to forward motion. We tackle the first via the bisegmental body so that θ˙b between the front and rear vector is distinct from the front vector’s θ˙or. Regarding the second, we introduce a simple linear bending-angle correction as the rear vector is aligned to the front vector’s orientation during forward motion, according to the equation : θb · (1 − d/dmax), if 0 ≤ d < dmax 0 θb = 0,

if d ≥ dmax dmax = l 2bc where d is the linear displacement during a timestep, b and b0 are the original and corrected bending angles, l is the body length and bc is the bend correction coefficient which has been fitted from the empirical data. In Fig. 8 E the resulting Δθb correction during individual strides is plotted in relation to θb at the beginning of the stride. The fitted linear curves show close matching of empirical and simulated findings.

To calibrate the turner oscillator we need to determine 5 parameters, namely the torque coefficient ct, the angular damping ratio z, the restorative spring constant k and the noise ratios nA, nT . To this end we analyze the angular dynamics of a population of 200 larvae and compute the distribution of 3 empirical metrics, namely bending angle θˆb, bend and ˙ ˙ orientation angular velocities θˆb, θˆor. In order to find optimal parameter combinations providing the best fit of simulated and empirical distributions a two-step parameter space search is performed : • We first evaluate the function of the isolated turner oscillator, in autonomous runs. We constrain the parameter space by selecting parameter combinations that fit the range of the empirical metrics (first and fourth quartiles within the empirical lower and higher halfs) without θb overshooting the maximum allowed π (data not shown). • We then evaluate the turner-crawler integration in free exploration simulations of virtual larva populations optimizing for best fit to the empirical distributions. The results of the optimal parameter set is shown in Fig. 8 D. Crawler-turner coupling. In order to reproduce bending behavior during strides we define a phase interval Prange during which the turner is free to exhibit its full effect on the body. Outside this interval, the torque T generated by the turner is attenuated by a scalar coefficient cat before being applied to the body, according to the equation :

T =

T , cat · T , otherwise

if pc in Prange where pc is the instantaneous phase of the crawling stride.

We calibrate the two parameters Prange and cat so that the average orientation angular velocity θ˙or during the stride cycle π fits the empirically measured. The results for Prange = [ 2 , π] and cat = 0.1 are shown in Fig. 8 A,bottom. Furthermore the orientation change during individual strides for the front Δθor and rear Δθorr segments fits well to empirical measurements (Fig. 8 C).

Olfactory sensor. Olfaction is introduced in the second layer of the control architecture allowing chemotactic behavior. The olfactory sensor is located at the front end of the virtual larva therefore any reorientation and/or displacement influences sensory input. As in ( 14 ) we assume that olfactory perception Ao relates to changes in odor concentration C according to the

Sakagiannis et al.

Weber-Fechner law, meaning that ΔAo ∼ ln ΔC. We further add a decay term that slowly resets Ao back to 0. The rate of change is given by the equation : i

˙ X Gi · Ci

Ci A˙o = −Ao · co + with − 1 ≤ Ao ≤ 1 [4] where co = 1 is the olfactory decay coefficient, Gi is the gain for odor i and Ci the respective odor concentration. Perceived olfactory stimulation Ao modulates the turner activation At from its baseline value At = 20 within a suitable range Arange = [Atmin, Atmax] = [10, 40] :

At = A¯t + Ao Atlim − A¯t where

Atlim =

Atmax Atmin ifAo ≥ 0 ifAo ≤ 0 [5] [6]

Parameter definition.

Segmentation and angular metrics. To specify the body segmentation providing the most suitable contact/rotation point for the definition of the correlated empirical bending θˆ˙b and orientation θˆ˙or angular velocities we analyse their relationship in a subset of 40 larvae. Tracking of 12 midline points allows computation of the absolute orientation of 11 body-segments and the respective 10 angles θ1−10 between successive body segments (Fig. 2 F). We define θˆor as the head-segment orientation because this defines the movement orientation of the animal. We ask how θˆ˙or results from the bending of the body as this is captured by the 10 angular velocities θ˙1−10. The regression analysis depicted in Fig. 9 B shows as expected that θˆ˙or depends primarily on the front angular velocities 0.22

Forward velocity. To define forward velocity we need to choose which midline-point is most suitable to track and which velocity metric to use for defining the start and end of a stride. To this end we perform stride annotation of 3-minute tracks of a population of 20 larvae using each of 24 candidate instantaneous velocity metrics, namely the velocities of the 12 points, the component velocities of the rearest 11 points parallel to their front segment’s absolute orientation and finally the centroid velocity. To compare the candidate metrics we compute the spatial cvs and temporal cvt coefficient of variation of the annotated strides for each larva to assess how variant their time duration and displacement is. We finally compute the mean cvs and cvt across individuals. In Fig. 9 A the spatiotemporal stride variance is shown for each candidate metric. We choose the metric that provides the minimal spatial and temporal stride variance, assuming that strides of an individual larva are more or less stereotypical in both duration and displacement ( 23 ). Our study reveals that the centroid velocity is the most suitable metric for stride annotation. All spatial metrics are therefore computed via this point’s displacement.

The instantaneous body-length of an individual larva fluctuates due to subsequent stretching and contraction during crawling. Its histogram is well fitted by a Gaussian distribution (data not shown). Therefore individual larva length l was computed as the median of the midline length across time (total length of the line connecting all 12 midline points). All spatial parameters, including displacement and velocity, are scaled to this body-length.

To analyze the temporal dynamics of crawling we perform spectogram analysis of the linear velocity. The dominant frequency fc across a plausible range of 0.75 to 2.5 Hz is defined as the crawling frequency. We use the inverse of this frequency fc−1 as a reference for the expected stride duration during stride annotation.

Epoch annotation. Strides are annotated using the forward velocity v timeseries, under a number of constraints (Fig. 2 C): • Each stride epoch is contained between two v local minima. • The v local maxima contained in the epoch needs to be higher than the mean. • The duration of the epoch needs to be between 0.6 and 1.6 times the reference stride duration tstr = fc−1, where fc the crawling frequency.This allows individual strides to temporally vary without overlapping so that adjacent strides can be concatenated in stridechains.

Stridechains are defined as uninterrupted sequences of successive strides (Fig. 2 C). Stridechain length equals the number of concatenated strides and is a discrete metric equivalent to crawl-run duration. Pauses are defined as epochs containing no strides. After stride annotation the resulting displacement due to each individual stride is computed for each larva and divided by the larva’s body-length (Fig.2 A-C). The individual 0.24 Fig. 9. Segmentation and velocity definition. A: Forward velocity definition. 24 candidate velocity metrics are compared for use in stride annotation of 3-minute tracks of a population of 20 larvae. For each candidate the mean coefficient of variation of temporal duration cvt and spatial displacement cvs of the annotated strides is shown. The centroid velocity vcen provides the most temporally and spatially stereotypical strides, therefore it is selected as the reference forward velocity for stride annotation and model fitting. vcen : centroid velocity, v1 − v12 : 1st-12th point’s velocity, v2c − v1c2 : 2nd-12th point’s component velocity parallel to the front segment’s orientation vector. B: Regression analysis of individual and cumulative angular velocities ˙ ˙ ˙ θi=1−10 to orientation angular velocity θˆor. When considered individually, θ2 best predicts reorientation with the θ˙1 and θ˙3 following. When considered cumulatively the anterior 5 θ˙i allow optimal prediction of reorientation velocity. C: Correlation analysis ˙ 5 of the sum of all possible θ˙i combinations to θˆor. The sum P θ˙i shows the highest i=1 5 correlation therefore we define θˆb = P θi as shown in A. For illustration purposes i=1 only the 5 highest correlations are shown. while this dependence decays as we move towards the rear segments, in line with previous studies ( 41 ). Timeshift analysis also shows that the front 3 angles change concurrently while angles further down the midline are increasingly lagging behind (data not shown). The correlation analysis depicted in Fig. 9 C shows that the sum of the front 5 angular velocities best correlates to θˆ˙or. In other words the cumulative body bend of the front 5 segments best predicts head reorientation. Therefore we define the reorientation-relevant bending angle 5 as θˆb = Pi=1 θi (Fig. 2 F). The remaining 5 angles between 12 | distributions are well fitted by Gaussians (data not shown). Therefore stride displacement dstr is defined as the average scaled displacement per stride for each larva.

Turn epochs are contained between pairs of successive sign changes of orientation angular velocity θo˙r. For each epoch the turning angle θturn is defined as the absolute total change of orientation angle Δθor (Fig. 2 G).

Supplementary videos. Here we provide the supplementary videos cited in the text. Videos can be played by clicking on the image frame.

Video 1. Bisegmental larva-body simplification The first video shows the original larva body as recorded by the tracker. 12 points are tracked along its longitudinal axis defining 11 segments while 22 points constitute the body contour. In the second video the body contour is dropped. In the third video an artificial rectangular contour is added for each body segment. In the last video the body-midline is segmented into 2 segments. The absolute head orientation angle θˆor is preserved while the single 5 bending angle between the 2 segmentw is defined as θˆb = Pi=1 θi

Video 3. Full-length larva trajectory The trajectory depicted in Fig. 2 is shown in its

˙ full length. Trajectory color tracks instantaneous forward v and angular θor velocities while tracking a midline point or the head respectively. Color code ranges from red (0 velocity) to green (maximum velocity).

Video 4. Larva trajectory slice The trajectory slice depicted in the inset of Fig. 2 is

˙ shown. Trajectory color tracks instantaneous forward v and angular θor velocities while tracking a midline point or the head respectively. Color code ranges from red (0 velocity) to green (maximum velocity).

Video 2. Locomotory model for Drosophila larva The function of the locomotory model in Fig. 1 B is illustrated by gradually integrating its 4 modules (crawler, turner, oscillator-coupling, crawling-intermittency). In each of the 6 videos the implemented modules are shown in the inset.

ACKNOWLEDGMENTS. This project was funded by the German Research Foundation (DFG) through a stipend for P.S. within the

Sakagiannis et al. Video 5. Free exploration simulation A population of 25 real (left) or virtual (right)

larvae is placed on a dish and left to freely explore. The body of the real larvae has been simplified into 2 segments as shown in Video 2(see Model, Bisegmental body). Research Training Group ‘Neural Circuit Analysis’ (DFG-RTG 1960, grant no. 233886668) and through the Research Unit ‘Structure, Plasticity and Behavioral Function of the Drosophila mushroom body’(DFG-FOR 2705, grant no. 403329959 to MN). We would like to thank Michael Schleyer and Juliane Thoener (Leipzig Institute for Neurobiology, Magdeburg, Germany) for providing the tracking dataset and for valuable discussions on the data analyses. We thank Bertram Gerber valuable discussion. We thank Ms Morozova for substantial support with the video and figure generation.

Video 6. Dispersion simulation The temporal course of dispersion from origin is Video 9. Filter selection The effect of inadequate or excessive filtering of the

shown for a population of 200 real (left) and virtual (right) larvae (Fig. 5 A).

Trajectories have been transposed to initiate from the same origin at the center of the screen. Video 7. Chemotaxis simulation The 2 simulations depicted in Fig. 6 are shown. In

the first, larvae are placed on the right side of the arena and are expected to approach an odor source located on the left side. In the second, larvae are placed in the proximity of an odor source located at the center of the arena and are expected to perform localized exploration, generating trajectories across and around the odor source.

Video 8. Odor preference simulation Two sample odor preference simulations are shown. Either an appettitive or an aversive odor source is placed on the left side of the dish while a non-valenced odor source is placed on the right. Larvae are left to explore and potentially navigate along the respective salient concentration gradient. 14 | empirical larva recordings is illustrated. The left video shows the jittery original recording while the effect of lowpass filtering at cuttoff frequencies of 4 Hz, 2 Hz and 0.5 Hz is shown on the rest. Selection of an intermediate 2 Hz cutoff frequency eliminates the unrealistic jitter while preserving the behaviorally relevant crawling frequency.

1. T Ohyama , et al., High-throughput analysis of stimulus-evoked behaviors in drosophila larva reveals multiple modality-specific escape strategies . PLOS ONE 8 , 1 - 21 ( 2013 ). 2. B Risse , D Berh , N Otto , C Klämbt , X Jiang , FIMTrack: An open source tracking and locomotion analysis software for small animals . PLoS Comput. Biol . 13 , 1 - 15 ( 2017 ). 3. I Schumann , T Triphan , The pedtracker: An automatic staging approach for drosophila melanogaster larvae . Front Behav Neurosci . 14 ( 2020 ). 4. D Tadres , M Louis , PiVR: An affordable and versatile closed-loop platform to study unrestrained sensorimotor behavior . PLoS Biol . 18 , 1 - 25 ( 2020 ). 5. MJ Almeida-Carvalho , et al., The Ol1mpiad : Concordance of behavioural faculties of stage 1 and stage 3 Drosophila larvae . J. Exp. Biol . 220 , 2452 - 2475 ( 2017 ). 6. S Gerhard , I Andrade , RD Fetter, A Cardona, CM Schneider-Mizell, Conserved neural circuit structure across drosophila larval development revealed by comparative connectomics . Elife 6 , 1 - 17 ( 2017 ). 7. T Jovanic , Studying neural circuits of decision-making in Drosophila larva . J. Neurogenet . 34 , 162 - 170 ( 2020 ). 8. D Kim , M Alvarez , LM Lechuga , M Louis , Species-specific modulation of food-search behavior by respiration and chemosensation in Drosophila larvae . Elife 6 , 1 - 23 ( 2017 ). 9. RK Vijendravarma , S Narasimha, TJ Kawecki, Predatory cannibalism in Drosophila melanogaster larvae . Nat. Commun . 4 , 1788 - 1789 ( 2013 ). 10. DW Sims, NE Humphries, N Hu , V Medan , J Berni , Optimal searching behaviour generated intrinsically by the central pattern generator for locomotion . Elife 8 : e50316 ( 2019 ). 11. P Sakagiannis , M Aguilera, MP Nawrot , A plausible mechanism for drosophila larva intermittent behavior in Biomimetic and Biohybrid Systems . (Springer International Publishing, Cham), pp. 288 - 299 ( 2020 ). 12. T Ueno , N Masuda , S Kume , K Kume , Dopamine Modulates the Rest Period Length without Perturbation of Its Power Law Distribution in Drosophila melanogaster . PLoS One 7 ( 2012 ). 13. AM Reynolds , et al., Evidence for a pervasive 'idling-mode' activity template in flying and pedestrian insects . R. Soc. Open Sci. 2 ( 2015 ). 14. A Gomez-Marin , GJ Stephens, M Louis , Active sampling and decision making in Drosophila chemotaxis . Nat. Commun . 2 , 410 - 441 ( 2011 ). 15. M Schleyer , et al., A behavior-based circuit model of how outcome expectations organize learned behavior in larval Drosophila . Learn. Mem . 18 , 639 - 653 ( 2011 ). 16. M Schleyer , et al., The impact of odor-reward memory on chemotaxis in larval Drosophila . Learn. Mem . 22 , 267 - 277 ( 2015 ). 17. B Gerber , RF Stocker, The drosophila larva as a model for studying chemosensation and chemosensory learning: A review . Chem. Senses 32 , 65 - 89 ( 2007 ). 18. S Diegelmann , B Klagges , B Michels , M Schleyer , B Gerber , Maggot learning and synapsin function . J. Exp. Biol . 216 , 939 - 951 ( 2013 ). 19. A Widmann , K Eichler , M Selcho , AS Thum , D Pauls , Odor-taste learning in Drosophila larvae . J. Insect Physiol . 106 , 47 - 54 ( 2018 ). 20. Q Wu , et al., Developmental control of foraging and social behavior by the Drosophila neuropeptide Y-like system . Neuron 39 , 147 - 161 ( 2003 ). 21. G Denisov , T Ohyama , T Jovanic , M Zlatic , Model-based detection and analysis of animal behaviors using signals extracted by automated tracking . Proc. Int. Conf. on Bio-inspired Syst. Signal Process ., 175 - 181 ( 2013 ). 22. D Karagyozov , M Mihovilovic Skanata , A Lesar , M Gershow , Recording neural activity in unrestrained animals with three-dimensional tracking two-photon microscopy . Cell Reports 25 , 1371 - 1383 . e10 ( 2018 ). 23. ES Heckscher, SR Lockery, CQ Doe, Characterization of Drosophila Larval Crawling at the Level of Organism, Segment, and Somatic Body Wall Musculature . J. Neurosci . 32 , 12460 - 12471 ( 2012 ). 24. A Wystrach , K Lagogiannis , B Webb , Continuous lateral oscillations as a core mechanism for taxis in Drosophila larvae . Elife 5 ( 2016 ). 25. G Ruiz-Dubreuil , B Burnet , K Connolly , P Furness , Larval foraging behaviour and competition in Drosophila melanogaster . Hered. (Edinb) . 76 , 55 - 64 ( 1996 ). 26. C Mantziaris , T Bockemühl , A Büschges , Central Pattern Generating Networks in Insect Locomotion . Dev. Neurobiol. 00 , 1 - 15 ( 2020 ). 27. A Miroschnikow , et al., Convergence of monosynaptic and polysynaptic sensory paths onto common motor outputs in a Drosophila feeding connectome . Elife 7 , 1 - 23 ( 2018 ). 28. M Thane , V Viswanathan , TC Meyer, E Paisios, M Schleyer , Modulations of microbehaviour by associative memory strength in Drosophila larvae . PLoS One 14 , e0224154 ( 2019 ). 29. B Szigeti , A Deogade , B Webb , Searching for motifs in the behaviour of larval Drosophila melanogaster and Caenorhabditis elegans reveals continuity between behavioural states . J. R. Soc. Interface 12 ( 2015 ). 30. KR Kaun , et al., Natural variation in food acquisition mediated via a Drosophila cGMPdependent protein kinase . J. Exp. Biol . 210 , 3547 - 3558 ( 2007 ). 31. G Slater , P Levy , KL Andrew Chan, C Larsen , A central neural pathway controlling odor tracking in drosophila . J. Neurosci. 35 , 1831 - 1848 ( 2015 ). 32. M Klein , et al., Exploratory search during directed navigation in C. elegans and Drosophila larva . Elife 6 , 1 - 14 ( 2017 ). 33. A Gomez-Marin , M Louis , Active sensation during orientation behavior in the Drosophila larva: More sense than luck . Curr. Opin. Neurobiol . 22 , 208 - 215 ( 2012 ). 34. E Paisios , A Rjosk , E Pamir , M Schleyer , Common microbehavioral "footprint" of two distinct classes of conditioned aversion . Learn. Mem . 24 , 191 - 198 ( 2017 ). 35. B Gerber , et al., Pain-relief learning in flies, rats, and man: Basic research and applied perspectives . Learn. Mem . 21 , 232 - 252 ( 2014 ). 36. M Schleyer , M Fendt , S Schuller , B Gerber , Associative learning of stimuli paired and unpaired with reinforcement: evaluating evidence from maggots, flies, bees, and rats . Front. psychology 9 , 1494 ( 2018 ). 37. N Koseki , et al., Individual differences in sensory responses influence decision making by Drosophila melanogaster larvae on exposure to contradictory cues . J. Neurogenet . 30 , 288 - 296 ( 2016 ). 38. T Niewalda , I Jeske , B Michels , B Gerber, 'Peer pressure' in larval Drosophila? Biol. Open 3 , 575 - 582 ( 2014 ). 39. TJ Prescott , P Redgrave , K Gurney , Layered control architectures in robots and vertebrates . Adapt. Behav . 7 , 99 - 127 ( 1999 ). 40. R Brooks , A Robust Layered Control System For A Mobile Robot . IEEE J. Robot. Autom. RA-2 , 14 - 23 ( 1986 ). 41. S Lahiri , et al., Two alternating motor programs drive navigation in Drosophila larva . PLoS One 6 ( 2011 ). 42. B Michels , et al., Pavlovian conditioning of larval Drosophila: An illustrated, multilingual, hands-on manual for odor-taste associative learning in maggots . Front. Behav. Neurosci . 11 , 1 - 6 ( 2017 ). 43. AM Jürgensen , A Khalili , MP Nawrot, Reinforcement-mediated plasticity in a spiking model of the drosophila larva olfactory system . BMC Neurosci . 2019 20( Suppl 1 ): P225 , 56 ( 2019 ). 44. AM Jürgensen , A Khalili , E Chicca , G Indiveri , MP Nawrot, A neuromorphic model of olfactory processing and sparse coding in the drosophila larva brain . bioRxiv ( 2021 ). 45. T Saumweber , et al., Functional architecture of reward learning in mushroom body extrinsic neurons of larval drosophila . Nat. communications 9 , 1 - 19 ( 2018 ). 46. D Owald , S Waddell , Olfactory learning skews mushroom body output pathways to steer behavioral choice in drosophila . Curr. opinion neurobiology 35 , 178 - 184 ( 2015 ). 47. D Owald , et al., Activity of defined mushroom body output neurons underlies learned olfactory behavior in drosophila . Neuron 86 , 417 - 427 ( 2015 ). 48. A Weiglein , F Gerstner , N Mancini , M Schleyer , B Gerber , One-trial learning in larval Drosophila . Learn. Mem . 26 , 109 - 120 ( 2019 ). 49. H Kohsaka , et al., Regulation of forward and backward locomotion through intersegmental feedback circuits in Drosophila larvae . Nat. Commun . 10 , 2654 ( 2019 ). 50. AA Zarin , B Mark , A Cardona, A Litwin-Kumar, CQ Doe, A multilayer circuit architecture for the generation of distinct locomotor behaviors in Drosophila . Elife 8 ( 2019 ). 51. J Berni , Genetic dissection of a regionally differentiated network for exploratory behavior in drosophila larvae . Curr. Biol . 25 , 1319 - 1326 ( 2015 ). 52. J Berni , SR Pulver , LC Griffith, M Bate , Autonomous circuitry for substrate exploration in freely moving drosophila larvae . Curr. Biol . 22 , 1861 - 1870 ( 2012 ). 53. T Zhang , A Branch, P Shen , Octopamine-mediated circuit mechanism underlying controlled appetite for palatable food in Drosophila . Proc. Natl. Acad. Sci . 110 , 15431 - 15436 ( 2013 ). 54. CA Malloy, et al., Pharmacological identification of cholinergic receptor subtypes: modulation of locomotion and neural circuit excitability in Drosophila larvae . Neuroscience 411 , 47 - 64 ( 2019 ). 55. C Eschbach , et al., Recurrent architecture for adaptive regulation of learning in the insect brain . Nat. Neurosci . 23 , 544 - 555 ( 2020 ). 56. K Vogt , et al., Internal state configures olfactory behavior and early sensory processing in Drosophila larvae . Sci. Adv . 7 , 1 , eabd6900 ( 2021 ). 57. M Schleyer , et al., Identification of dopaminergic neurons that can both establish associative memory and acutely terminate its behavioral expression . J. Neurosci . 40 , 5990 - 6006 ( 2020 ). 58. A Schoofs , et al., Selection of Motor Programs for Suppressing Food Intake and Inducing Locomotion in the Drosophila Brain . PLoS Biol . 12 ( 2014 ). 59. CS Qian, M Kaplow , JK Lee , WB Grueber, Diversity of internal sensory neuron axon projection patterns is controlled by the POU-domain protein pdm3 in drosophila larvae . J. Neurosci. 38 , 2081 - 2093 ( 2018 ). 60. JS de Belle, M Heisenberg , Associative Odor Learning in Drosophila Abolished by Chemical Ablation of Mushroom Bodies . Sci. ( 80 -. ). 263 , 692 - 695y ( 1994 ). 61. A Strutz , et al., Decoding odor quality and intensity in the Drosophila brain . Elife 3 , e04147 ( 2014 ). 62. MJ Dolan , et al., Communication from Learned to Innate Olfactory Processing Centers Is Required for Memory Retrieval in Drosophila . Neuron 100 , 651 - 668 . e8 ( 2018 ). 63. C Eschbach , et al., Circuits for integrating learnt and innate valences in the fly brain . Bioarxiv , 1 - 72 ( 2020 ). 64. I Tastekin , et al., Sensorimotor pathway controlling stopping behavior during chemotaxis in the Drosophila melanogaster larva . Elife 7 , 1 - 38 ( 2018 ). 65. J Gjorgjieva , J Berni , JF Evers, SJ Eglen , Neural circuits for peristaltic wave propagation in crawling drosophila larvae: Analysis and modeling . Front. Comput. Neurosci. 7 , 1 - 19 ( 2013 ). 66. C Pehlevan , P Paoletti , L Mahadevan, Integrative neuromechanics of crawling in D. melanogaster larvae . Elife 5 , 1 - 23 ( 2016 ). 67. D Ross , K Lagogiannis , B Webb , A model of larval biomechanics reveals exploitable passive properties for efficient locomotion . Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics ) 9222 , 1 - 12 ( 2015 ). 68. J Loveless , K Lagogiannis , B Webb , Modelling the neuromechanics of exploration and taxis in larval Drosophila . PLoS Comput. Biol . 15 , 7 ( 2019 ). 69. A Davies , M Louis , B Webb , A Model of Drosophila Larva Chemotaxis . PLoS Comput. Biol . 11 , 1 - 24 ( 2015 ). 70. MN Günther, G Nettesheim, GT Shubeita, Quantifying and predicting Drosophila larvae crawling phenotypes . Sci. Rep . 6 , 1 - 10 ( 2016 ). 71. M Zago , F Lacquaniti , A Gomez-Marin , The speed- curvature power law in Drosophila larval locomotion . Biol. Lett . 12 , 3 - 6 ( 2016 ). 72. RS Marken , DM Shaffer, The power law of movement : an example of a behavioral illusion . Exp. Brain Res . 235 , 1835 - 1842 ( 2017 ). 73. M Zago , A Matic , T Flash , A Gomez-Marin , F Lacquaniti , The speed - curvature power law of movements: a reappraisal . Exp. Brain Res . 236 , 69 - 82 ( 2017 ). 74. RS Marken , DM Shaffer, The power law as behavioral illusion: reappraising the reappraisals . Exp. Brain Res . 236 , 1537 - 1544 ( 2018 ). 75. E Bicho , Ph .D. thesis (University of Minho) ( 1999 ). 76. PM Itskov , et al., Automated monitoring and quantitative analysis of feeding behaviour in Drosophila. Nat. Commun . 5 ( 2014 ). 77. N Otto , et al., Interactions among Drosophila larvae before and during collision . Sci. Rep . 6 , 1 - 11 ( 2016 ). 78. JB Masson, et al., Identifying neural substrates of competitive interactions and sequence transitions during mechanosensory responses in drosophila . PLOS Genet . 16 , 1 - 40 ( 2020 ). 79. T Jovanic , et al., Neural Substrates of Drosophila Larval Anemotaxis. Curr. Biol . 29 , 554 - 566 . e4 ( 2019 ). 80. T Sokabe , HC Chen, J Luo , C Montell , A Switch in Thermal Preference in Drosophila Larvae Depends on Multiple Rhodopsins . Cell Rep . 17 , 336 - 344 ( 2016 ). 81. M Klein , et al., Sensory determinants of behavioral dynamics in Drosophila thermotaxis . Proc. Natl. Acad. Sci . U. S. A. 112 , E220 - E229 ( 2015 ). 82. L Luo , et al., Navigational decision making in Drosophila thermotaxis . J. Neurosci . 30 , 4261 - 72 ( 2010 ). 83. ML Zhu , KJ Herrera, K Vogt , A Bahl , Navigational strategies underlying temporal phototaxis in Drosophila larvae . bioRxiv ( 2020 ). 84. J Felsenberg , et al., Integration of Parallel Opposing Memories Underlies Memory Extinction. Cell 175 , 709 - 722 . e15 ( 2018 ). 85. C König , et al., Reinforcement signaling of punishment versus relief in fruit flies . Learn. Mem . 25 , 247 - 257 ( 2018 ). 86. AS Thum , B Gerber , Connectomics and function of a memory network: the mushroom body of larval drosophila . Curr. opinion neurobiology 54 , 146 - 154 ( 2019 ). 87. T Landgraf , et al., NeuroCopter: Neuromorphic computation of 6D ego-motion of a quadcopter in Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics) . Vol. 8064 LNAI, pp. 143 - 153 ( 2013 ). 88. T Saumweber , et al., Behavioral and synaptic plasticity are impaired upon lack of the synaptic protein SAP47 . J. Neurosci . 31 , 3508 - 3518 ( 2011 ). 89. B Michels , et al., Cellular site and molecular mode of synapsin action in associative learning . Learn. Mem . 18 , 332 - 344 ( 2011 ). 90. A Widmann , et al., Genetic Dissection of Aversive Associative Olfactory Learning and Memory in Drosophila Larvae . PLoS Genet . 12 , 1 - 32 ( 2016 ). 91. M Springer, MP Nawrot, A mechanistic model for reward prediction and extinction learning in the fruit fly . eNeuro ( 2021 ).