My PhD Project

...includes the development of a lightweight and low-cost quadrotor flight laboratory and studies nonverbal interaction methods in human-drone encounters.

Nonverbal Human-Drone Interaction

Publications

• Nonverbal Human-Drone Interaction, Marc Lieser, Doctoral Dissertation, RheinMain University of Applied Sciences, Wiesbaden, 2024. University and State Library RheinMain / PDF
• Vocalics in Human-Drone Interaction, Marc Lieser and Ulrich Schwanecke, 33rd International Conference on Robot and Human Interactive Communication (RO-MAN), 2024. Digital Library (IEEE) / PDF
• Evaluating Distances in Tactile Human-Drone Interaction, Marc Lieser, Ulrich Schwanecke, and Jörg Berdux, 30th International Conference on Robot and Human Interactive Communication (RO-MAN), 2021. Digital Library (IEEE) / PDF
• Tactile Human-Drone Interaction: MetroDrone, Marc Lieser, Ulrich Schwanecke, and Jörg Berdux, 15th International Conference on Tangible, Embedded and Embodied Interaction (TEI), 2021. Digital Library (ACM) / PDF
• EAVE: Emotional Aerial Vehicle Evaluator, Marc Lieser, Ulrich Schwanecke, and Jörg Berdux, INFORMATIK 2020, 2021. Digital Library (GI) / PDF
• A Low-Cost Mobile Infrastructure for Compact Aerial Robots Under Supervision, Marc Lieser, Henning Tjaden, Robert Brylka, Lasse Löffler, and Ulrich Schwanecke, European Conference on Mobile Robots (ECMR), 2017. Digital Library (IEEE) / PDF

Reviews

• 33rd International Conference on Robot and Human Interactive Communication (RO-MAN), 2024.
• 18th Annual ACM/IEEE International Conference on Human Robot Interaction (HRI), 2023.

Session Chairs

• Co-chair of the Sound Design for Robots session at the 33rd International Conference on Robot and Human Interactive Communication (RO-MAN), 2024.

»Nonverbal Human-Drone Interaction«

Over the last decade, miniature aerial vehicles (MAVs) evolved as the standard platform in aerial robotics research. Due to their mechanical simplicity, quadrotors in particular, i.e., drones with four rotors, are used in a wide variety of professional and consumer applications. Given the natural inhibition threshold, that humans experience when interacting with robots whose intentions are difficult for them to construe, and due to the limitations of MAVs to carry additional payloads, I am particularly interested in exploring different ways and new methods to interact nonverbally with drones.

With usually very high expenses for sophisticated testbeds, that are well equipped with costly technology, especially groups of smaller institutes are kept from getting started in the exciting field of MAVs. So we built our own based on consumer and hobby hardware. ICARUS, the low-cost mobile infrastructure for aerial robots under supervision, is a lightweight and affordable testbed for general research and applications in the field of aerial robotics. It consists of multiple radio remote control approaches, quadrotors from the hobby area and an optical monocular tracking system with our infrastructure consequently resorting to off-the-shelf hardware that is simple to replicate. The radius of action can easily be expanded by using multiple cameras with overlapping frustums.

A regular consumer notebook is capable of doing all the work. The tracked poses are fed into hover or model predictive path tracking controllers and the control variables are sent via serial connection to different Arduino-based radio remote control approaches or the Bitcraze Crazyradio PA. These small dimensions make our system also very portable, so it can easily be used for live demonstrations, for example as a part of a lecture or an exhibition. In the last years the Crazyflie quadrotor emerged as a very convenient development platform and its Crazyradio is also integrated into our infrastructure.

Currently, the indoor environment is mainly used for experiments with new gesture-based and tactile human-drone interaction scenarios to replace operating remote control levers with a more natural interaction. But to demonstrate the capability of the infrastructure, several applications were realized based on ICARUS. For example an experimental Search and Rescue (SAR) application was developed, where a quadrotor extends the limited field of view of an unmanned ground vehicle (UGV). For this, we transferred the whole infrastructure to a Raspberry Pi 3, tracking at 50 Hz with the official Raspberry Pi Camera Module. Another example is long-exposure photography, a technique also called light painting provides a good intuition for the accuracy of our control system.

Publications

See research overview page.

TRACKSHOT

»Tracking of Shotgun and Target for Olympic Trap and Skeet«

A project in the field of performance diagnostics in cooperation with Deutscher Schützenbund e.V..

My part in this project was building and programming a suitable camera rig, which included the selection of cameras/lenses, designing and soldering the circuit to enable hardware triggered, synchronized exposures and implementing the software to acquire and process images. A mid 2012 MacBook Pro is capable of capturing images from three cameras (two 2.2 megapixels color images and one 1.3 megapixels grayscale image) at a constant rate of 156 frames per second. I decided to implement a circular buffer to queue the images until the user (e.g., an olympic coach) decides to either manually analyze the athlete's process of targeting and shooting or to start the computer vision-based evaluation of the captured streams.

Since it is not possible to attach a single tracking marker to the shotgun barrel in such a way that it is aligned exactly parallel to the barrel, accurate calibration of the barrel to the marker was another challenge. Just a slight deviation in the angle can account for a large error in the distance at which the clay targets fly. The developed system prototypically consists of a round profile, which is attached to an empty shotgun cartridge. As with the loading process, the shotgun is opened and the cartridge attached to the round profile is inserted into the chamber. A schematic representation of the calibration process is shown in the figure below.

After insertion of the calibration rod, it is rotated carefully with some pressure on the chamber, causing the determined measuring points of two additional markers to describe two circular paths. Since the markers used differ from each other due to different LED arrangements, the measuring points can be clearly assigned. A line through the centers of the circles then describes the shotgun barrel direction in relation to the marker that is permanently attached to the shotgun.

TILT

»Trifocal Cylinder Tracking«

Method and hardware design to determine the orientation of a cylindrical object from the view of up to three cameras.

Traditional computer vision algorithms are used to detect the edges of the cylinder. A singular, symmetrical matrix of the degenerate conic is calculated from the two line equations of the cylinder edges. Using the singular values and the associated eigenvectors of this matrix and the given cylinder radius, the Plücker coordinates of the cylinder axis can be determined. The result can be improved by incorporating the axis measurements from the images of the additional cameras.

Projection-ONE

»Structured Light 3D Scanner«

Computer-aided 3D scanning methods make it possible to produce detailed digital and spatial images of complete real scenes and individual objects. These scans are used in many areas: 3D objects are created in the gaming and entertainment industries, quality management in industrial manufacturing detects cracks and other production flaws, and computer tomographies are generated in the medical field.

In computer-aided scanning procedures, a distinction is made between active and passive procedures. In passive methods, monocular or stereoscopic photos or videos are taken of the scenes to be scanned. From these, the spatial structure is calculated without enhancing the scenes themselves with additional information (sensors, markers or light patterns, etc.). In active methods, on the other hand, information is added to the scenes for the scanning process. This includes, for example, laser scanning processes. These methods usually deliver much more precise and detailed results than passive methods, but they require very expensive hardware and are therefore mainly used in an industrial context.

The Projection-ONE Scanner is a so-called Structured Light Scanner (active method) as described in [Bronstein et. al., High-Resolution Structured Light Range Scanner with Automatic Calibration, 2003]. At its core, the device consists of a monocular camera and a commercially available projector. In such a constellation, the acquisition costs are also affordable for private use.

FPV

FPV drones (first person view) are manually controlled compared to the more familiar stabilized drones, providing much more natural motion. With a total takeoff weight of less than 250 grams, FPV drones can be flown close to people, and their compact design and manual control allow for unprecedented perspectives.

The quadrotor simulation I had implemented as part of my PhD project was easily extended to a realistic FPV simulation. Developing and testing the simulation involved many hours of flying in cell-shaded OpenGL/glsl parts of cities like New York created from OpenStreetMap data or low-poly worlds modeled with Blender. I never had (and still have) as much fun as when I finally went out into the real world in 2018 or so. I think I was lucky enough to skip the frustrating part that sometimes comes with such a sophisticated hobby through the many hours I spent in simulation.

The video of the FPV simulation below is a bit older. Meanwhile the visualization has improved a lot (hdr, bloom, order-independent transparency) since I used my engine for the 3D renderings in my thesis, but I did not find the time to render a new video.

In 2022, I had the opportunity to race the Hockenheimring (video below), where my dad took our family to DTM races when we were kids. Not with a car, but with a drone; almost race track and almost in the right direction!

For FPV inquiries please use the form at mediamarc.de!

3D Print

Here is a selection of 3D prints I designed:

Bathroom Ladder Radiator Towel Hook

Vinyl Stand

Marshall Cabinet Side Handle Instrument Cable Holder

More models I designed can be found on Printables.

Tinkering With Light and Shadows

I love that we live in a time when microcontrollers, LEDs, and 3D printers allow us to implement small, creative ideas ourselves in no time.

Examples of my work include the augmenting of an antique map of Paris with live solar and lunar positions, tuning my TV lowboard with ambient lighting, and building a visual equalizer.

Solar/Lunar Map

Poor Man's Ambilight

Visual Equalizer

Currently I am experimenting with shadow art and light painting using drones.

Shadow Art

Light Painting

Making Music

I play the guitar since I was a little kid. I started out with the old acoustic guitar from my mother and switched to the electric guitar at the age of twelve or thirteen. Shortly after, my friends and I founded a school punk/hardcore band. After open rehearsals in the school's auditorium, a few local gigs and one semi-professionally recorded song we disbanded. But the remaining three of us and two members of another local band that just parted ways decided to merge under the name Last Nights Favorite. We recorded a 5-track EP called After All These Years and drove 4436.8 km through Germany to play 21 gigs. We disbanded in 2008.

From three concerts that took place in 2007/2008 in Idar-Oberstein, Reil and Wiesbaden, I edited this video:

Shortly after the split I recorded some acoustic songs in my flat but then abandoned my instruments for the major part of my studies. Since my graduation I'm back to making music. I write/record/edit/mix own songs, put them in a shared directory with the intersection of members of my first two bands and we meet once a year to refine and record them at a final stage. It's pretty much a working long-distance rehearsalship. But since I was missing face-to-face discussions with other musicians about riffs and breaks in the atmosphere of a rehearsal room I just recently started a new local project.

However, this is an instrumental song I made that represents the state before going to rehearsal:

Illustration and Design

Not least because of the need for concert flyers, a band logo and a home page (optimized for IE6.0;) for my first band called NOCOMROMIZE I somehow came to graphics and a little web design. Back in the days we were a young punk/hardcore band founded in 9th grade. The hardcore scene was strongly bound to the DIY scene, attitudes I never laid off ever since. The designs may not have been the prettiest, but they worked and didn't cost anything.

Some years later I found myself designing logos and programming MySpace pages for bands I was friends with in my free time. A little later I started studying media computer science, where a minor part addressed design and I got the chance to be taught what I had been doing raggedly for quite a few years at that point.

However, music and bands have been the animating spirit for my design hobby as everything I did (flyers, posters, logos, websites, GIFs for banners, shirts, video editing) originally was related to that.

Vinyl

I bought my first batch of records as a kid on a flea market. I didn't even own a record player, as I was born five years after the CD, which by the time had already superseded a format believed dead. I was just fascinated by the mechanical fashion of the needle picking up the sound from the grooves. Apart from the sound quality in a halfway decent setup, the whole process is just a different way of perceiving music: selecting a record, placing it on the player, carefully lowering the needle, sinking into the sofa and consciously listening to music. Somewhere along the development of mp3 and streaming services, this experience has been lost and music far too often no longer receives the proper attention it deserves.

Though my collection is of a manageable size, it contains some Post-Hardcore gems. Unfortunately, a lot of albums which mean the world to me were produced in the first decade of the 21st century—a time when good vinyl engineers were hard to find. It's nice to see this format coming back to life and even overtaking the CD again.

Snowboarding

Snowboarding is my freedom. A cold day spent riding in the Dolomite Alps may have the power of clearing my mind of everything. A starry night in snow silence that absorbs what is left of noise is where time stands still and I am at peace. There's no place I'd rather be.

I'm really thankful that my parents took us kids on yearly skiing vacations and started this love.

»Visual Equalizer«

Equalizer Visualization using a LED matrix driven by an Arduino Uno. The sound is captured with a microphone in order for the device to not depend on a specific audio source.

Links

Source Code

• GitHub Repository

Hardware Components

• Arduino Uno
• LED Matrix
• Microphone

»Solar/Lunar Map«

I found this beautiful, mid-century Plan du Centre de Paris à Vol d'Oiseau (map of central Paris from a bird's-eye view) by Blondel la Rougery in my parents' attic.

First, I wanted to add some ambient lighting to it, but some time later I found myself reading books on astronomical calculations and expanding the map to show the positions of the Sun and Moon as seen from the geographic coordinates of the map. In addition to the solar and lunar positions, I added modes to display the current time in the style of an analog clock and some random animations, as well as the originally intended static ambient lighting.

The video below demonstrates all implemented modes. For details and instructions, please check the Readme located in the GitHub repository!

Demo

Links

Source Code

• GitHub Repository

Web Interface

• Boostrap/jQuery Website

Main Components

• ESP8266
• Neopixel RGBW LED Strips
• Power Supply RSP-75-5

Media Coverage

• "Antique Map of Paris with Modern Tech" —Bryan Cockfield, hackaday.com
• "Sonne, Mond und Neopixel" —Rebecca Husemann, Heise Make 4/2021 S.46

Pictures

»Poor Man's Ambilight«

I bought the lowboard for my TV second-hand. To my surprise, there was an LED strip (with non-addressable LEDs) integrated, but with a defective controller. I replaced it with an Arduino Nano and should have left it at that. But unfortunately, I have to implement some of my ideas to make room in my brain for something new. So I added a second strip behind the TV, expanded it with a color sensor, added a switch and a button, and moved the whole project from the Arduino to a microcontroller on a breadboard and put it in a case.

Usage

I wanted to keep it as simple as possible, so I added only a single button to control everything, that is to switch between dynamic light mode and static light mode and within static light mode to change the color using the Hue, Saturation, and Value (HSV) color model.

• A short press (< 250 ms) cycles through the current HSV parameter index in static mode,
• A long press (> 1 s) sweeps through brightness in ambient mode or through the selected HSV parameter in static mode.
• In between presses toggle between static and ambient modes.

Links

Source Code

• GitHub Repository

Hardware Components

• Microcontroller ATMEGA 328P-PU
• RGB Color Sensor TCS34725
• Power Supply 12V 3A
• Push Button T 113A SW
• Power Switch SPDT On-On
• Color Sensor LED Switch SPDT On-Off-On (intended for experimentation with the color sensor; should be skipped)
• DC/DC Converter TSR 1-2450
• MOSFETs IRLB 8721 (3)
• Crystal Oscillator IQD LFXTAL003240
• 22 pF Crystal Oscillator Capacitors KERKO-500 22P
• 100 nF Decoupling Capacitor KERKO 100N
• DC Barrel Socket 5.5 mm / 2.1 mm
• A 72x50x26 mm³ case (I had no printer at the time; you definitely should go larger)

Media

»Shadow Art«

This was a test design and print for an idea that is still developing.

»Light Painting«

Using our drone infrastructure to create light paintings for Christmas cards.