Biorobotics Laboratory BioRob

Powerful deep-learning based tracking method for animal behaviors requires large-scale curated and annotated data. Several recent papers [1,2] revealed the possibility to leverage the data requirement by rendering animated synthetic animals such as mice and ants.

In this project, the student will work on an existing biomechanical model of Salamander Pleurodeles Waltl to create a synthetic dataset for marklerless keypoint tracking tasks. The dataset would help improve the performance of a salamander tracking network, which would ultimately provide invaluable kinematics data for designing muscle models, neural controllers and validating neuroscience hypothesis.

For a PdM, this project involves:

• Improve the realism of the current salamander model in Blender and add diversity with procedurally generated noise and domain randomization.
• Generate a synthetic image dataset for markerless tracking tasks.
• Train an Image Domain Translator (e.g. U-GAT-IT[3]) to increase the dataset fidelity and reduce the reality gap
• Bonus: evaluate the dataset power on a markerless tracking network (e.g. DLC[4])

For a semester project, work packages will be optionally dropped and tailored according to student’s skills and interests.

The student is expected to have good programming skills and previous experience/knowledge in deep learning. Knowledge in 3D modeling and computer graphics is a plus but not required. If interested, please send an email to Chuanfang Ning with your motivation, CV, transcripts and most relevant experience.

[1] Bolaños, Luis A., et al. "A three-dimensional virtual mouse generates synthetic training data for behavioral analysis."

[2] Plum, Fabian, et al. "replicAnt: a pipeline for generating annotated images of animals in complex environments using Unreal Engine." 

[3] Kim, Junho, et al. "U-gat-it: Unsupervised generative attentional networks with adaptive layer-instance normalization for image-to-image translation." 

[4] Mathis, Alexander, et al. "DeepLabCut: markerless pose estimation of user-defined body parts with deep learning." 

Mobile robots often communicate over the 2.4 GHz band using standard off-the-shelf technologies as WiFi or Bluetooth, or sometimes custom radio protocols either on the 2.4 GHz or 868 MHz ISM bands, both on the UHF part of the radio spectrum. This project aims at evaluating the possibility of using the 169.4 MHz band (VHF) for controlling robots and obtaining telemetry, as it might give much better results in terms of range and transmission through obstacles or water, even if the available bandwidth is much more restricted.

The project involves:

• Identifying an appropriate RF module/chip
• Creating a printed circuit if necessary
• Using a microcontroller to control the RF module and obtain bidirectional communication
• Experiments for range in open air, through obstacles and underwater

Requirements: experience with digital electronics and basic understanding of radio communications and related concepts (e.g. transmission lines, antennas). Previous experience with radio frequency and/or PCB design is a plus.