By: Prof. Shlomo Maital

A praying mantis, an owl, a pigeon and a chameleon go into a bar. The praying mantis pulls out a Bible and says to the barman…

Actually, this is not the start of a bad joke. And these creatures didn’t go into a bar. They were brought into Technion Computer Scientist Ehud Rivlin’s robotics lab. Prof. Rivlin is an expert on robotic machine vision. Helping robots see better, it turns out, is a key part of improving how they function. Over 20 years ago, Rivlin embarked on a risky and challenging research project, to better understand vision by studying creatures who see far better than humans. His quest brought him to study birds, insects and lizards. His goal: Make robots that can see much better. And he and his collaborators have succeeded.

Caltech Jet Propulsion Laboratory expert Yoseph Bar-Cohen said, at a 2005 conference, “After billions of years of evolution, nature has learned what works and what would last.” Survival of the fittest has given praying mantises superb vision so they can snatch flies and mosquitoes out of the air; otherwise, they starve. Natural selection has given similarly sharp eyes to chameleons, owls and pigeons.

Learning from Nature is a large and growing field called “biomimetics,” which means imitating nature. Need tough cable? You could use steel threads. But it is far better to use spider silk, which has tensile strength twice as high as steel. But how do you milk spiders? A creative Montreal-based company called Nexia Biotechnologies worked the following magic: Genes from the spider were incorporated into the milk glands of goats. The resulting silk protein in the transgenic goats’ milk was then harvested and spun into silk fibers. It is not as good as real spider silk, but is still tougher than steel.

The most famous biomimetic invention is Velcro. In the late 1940s, a Swiss engineer, George de Mestral, took his dog for a walk. As happens to many of us dog lovers, he noticed cockleburs sticking to his dog’s fur. He analyzed the  tiny hooks that stick the seed-bearing burs to the dog’s coat and by 1955 had patented Velcro, a fastener with hooks on one side and soft loops on the other.

I asked Prof. Rivlin what led him to tackle a research topic that requires expertise in both computer science and biology. “What drove my research,” he said, “was three things. First, curiosity.  The fly has a very small brain, with limited number of neurons, yet it does very complex things, like flying and landing precisely on a certain spot. How? Why? I was curious why praying mantises see so well. Second, ‘existence proof.’ From my high school class emerged six or seven professors. Why? We were given hard problems in math, told there was a solution and asked to find it. The fact we knew there was a solution motivated us. If we thought there was no solution, we would have worked far less hard on it. So I wanted to prove the feasibility of learning about vision and optimizing machine vision, from insects and animals. And third, I wanted to research unconventional topics. To create change, to do things better. Research that combines biology and computer science is high risk, but also high reward.”

Some 20 years ago, Rivlin wrote a paper with colleagues at the University of Maryland, arguing that to understand vision in sharp-eyed creatures you need to study the purpose of that vision: food, mating, survival. This direction proved fruitful. What Rivlin has done is, for instance, to learn how the praying mantis tracks its prey by using high-speed stop-motion cameras. He then built a mathematical algorithm, or formula,  that can be used to build a mantis head “machine eye.” He also built a way to simulate ‘food’ for the mantis.

Why is the praying mantis so good at catching its prey? In a 2003 paper Rivlin and his student Igor Katsman (who is now launching a startup) found that it likes to move its head from side to side (a technique known as ‘peering,’ also giving it its name, because peering resembles praying). This creates “motion parallax,” a way to judge depth by the intersection of two different sight lines. The mathematical model Rivlin built to capture this technique was then incorporated into a robotic camera.

I recalled meeting a sculptor, Joe E. Brown, during my student days at Princeton. Brown had lost an eye while training as a boxer. For depth perception, you need two eyes. How could Brown sculpt without the ability to judge depth? He solved the problem creatively, by placing his subjects on a moving turntable. His single eye judged depth by seeing objects in two places (two sight lines), rather than having two eyes seeing an object in one place. He went on to become America’s leading sculptor of sports figures.

Pigeons, owls and chameleons also have superb vision, each in their own way. Pigeons bob their heads up and down  for the same reason the praying mantis ‘peers’ ‒ to create two intersecting sight lines on a scrap of food, so its peck will be fast and accurate. Owls’ eyes are fixed in their sockets, so the owl uses its highly flexible neck to revolve its head; owls spend a third of their time revolving their heads like radar. In contrast, the amazing chameleon has eyes that move independently in their orbits (unlike human eyes, which must move together). This helps the chameleon track its prey and “fire” its sticky tongue precisely to catch it, like an Iron Dome missile pinpointing an enemy rocket. Rivlin has studied all these creatures, learned the secrets of their sharp vision and works to incorporate them into his sharp-eyed robots.

I asked Prof. Rivlin about possible uses of improved machine vision. He mentioned “docking,” a crucial procedure which requires sharp vision used, for instance, in the International Space Station, or even asking a robot to plug an electric cord into a socket. The military, of course, is keenly interested in robots and in robotic vision. It could be helpful for observation drones.

During an Open House held by the Computer Science faculty, I saw a demonstration of a helicopter drone. Rivlin said that his lab is studying dragon flies, whose aerobatic capabilities far exceed those of helicopters, to improve those drones. Dragon flies have been around for 350 million years. The first helicopter flew in 1939, only 73 years ago. No wonder dragon flies fly better.

I asked Prof. Rivlin what he is currently researching. He said he is trying to create an improved ‘virtual reality’ for his creatures, to simulate ‘food’ so their behavior can be studied better. He is also trying to scale up his praying mantis research, so that seven at a time can be studied rather than just one.

How can help researchers like yourself connect better with real-world problems? I asked him. He has a simple answer. “Recently, at a workshop, doctors met with researchers and recounted the problems and challenges the doctors faced. This helped the researchers learn about real-world problems that might also be challenging and fascinating research topics.”

For nature lovers, be assured that Rivlin, his colleagues and students all treat the creatures they study with love and affection.

In a popular TV show in the 1970s, Bionic Man Steve Austin, the Six Million Dollar Man, fights evil with his bionic muscles and bionic left eye. Though Prof. Rivlin would not claim so, machine vision may one day help sightless people to see, or help sighted people to see even better. And again, science fiction will become reality, as it so often does.

This article was originally published in the Jerusalem Post Marketplace on January 14, 2013.