An automobile on auto-pilot? Just how do we plan on pulling that one off?
When we look out our windshield, we know where the road is. Though men are notorious for not stopping to ask for directions, we usually get where we are going and we rarely run into another car or a tree. We know how to avoid hitting other objects, at least most of the time. We don’t know how we know, we just know. That fact that we don’t fully understand how we know what is road and what is another car makes automating driving really hard. If we don’t know how we do something, how can teach a machine to perform the same task?
We already have most of the technology. The challenge is to put it all together. Though this is a very high-level overview, here’s what parts are involved:
Three little words: Global Positioning System (GPS.) Though there were various other efforts to provide location, GPS has had the most impact. The initial release was to enable two different capabilities. Civilians could get information about their position to within 50 meters, and the military could get their location to within 5 meters. In recent years, the 5-meter accuracy has been opened up to civilians. Problem solved? Not quite. 5 meters is about 15 feet. A parking spot is also about 15 feet, and so is the width of a lane. If you are off by 5 meters, you are either in someone else’s parking spot or your shiny new car is wrapped around a tree. GPS is good enough to tell you which road you are on, but not which lane you are in. The other problem is that GPS has to have clearance to the satellites in order to receive the signals needed. (GPS requires a direct line of sight to at least 3 satellites in order to obtain location.) Go in a tunnel, under trees, or between tall buildings and GPS goes away.
Dead reckoning has been used for ages. In the case of a car, multiply the number of times the wheel turns by the circumference of the tire and you know how far you have gone. The problem here is that wheels go different distances when the car goes around a turn, and there are various other reasons why this gets off after a while. (Dead reckoning is apparently a politically incorrect term. The new buzzword is Ground Truth. Go figure.)
If you have $20,000 to $100,000 to spend, you can buy a cool box that has a gyroscope and other goodies to help tell you where you are. Even this can place your car several meters off track. The technology is to where it can provide location within about 10 cm, which is amazing. However, that’s best case and is not there 100% of the time.
The car has to be equipped with sensors (usually video cameras) that can be used to find the road. The bottom line is that you have to use several pieces of technology – GPS, dead reckoning, and computer vision to keep the vehicle on the road.
The hard part is detecting obstacles. There are several options here. Two cameras can be used to generate a stereo map much like what people see using their two eyes. Two (or more) cameras are mounted so they are a little bit apart but look in the same direction. Software makes an attempt to find the same object in the video captured from both cameras at the same time. The difference in the location on the image can be used to compute the distance to the object. Human stereo vision only provides 3D information to our brains for about 15 feet. Beyond 15 feet, we use the size of object to judge distance. Cameras can sometimes do a little better, but they are far from perfect.
Software can also detect objects by color and texture. Texture is more computer intensive, and you don’t have a lot of time to spend on each frame from the camera. 30 MPH is 44 feet per second. A lot of collisions can happen in 44 feet if you are only processing the sensor data once a second. OK, only one collision happens, because your car is out of the race after that one. Color would be great if they would make all the roads green and all the obstacles bright red, but many cars are near the same shade as pavement. Software can usually detect moving objects, but this is more difficult because your car is also moving. Besides, what if the car you are about to hit is parked? If the car is stationary and about the same color as the road, the computer might decide it is ok to drive right through that new gray sedan you got for Christmas.
Radar has been used by some of the robotic cars, but it does not give the detailed local imagery needed. Besides, it is expensive and it is hard to get the data into the computer. Ultrasonic transducers provide another option, but this technology isn’t effective beyond a very short distance.
Almost all of the DARPA entries use LIDAR (laser light range finders) made by SICK. This provides a very detailed distance map of what is nearby. These units cost about $5,000+ each, and it really helps if you have several of them so break out the checkbook. They have their problems like anything else, but their use has been a major advance in the field of robotics.
This part isn’t easy, but is easier than the rest of the challenge. The break can be pushed by a linear actuator or configurations like a wench can pull the break peddle down. Throttle takes less force, and a servo can be used to pull a cable or another linear actuator can be used to push the gas peddle. An electric motor is normally attached to the steering wheel column. Stepper motors provide better control, but most do not have the torque required. A few simple relays to control the turn lights and other devices and away you go.
Many new cars are drive by wire, meaning everything is computer controlled. Instead of pushing the break and gas and turning the wheel, you simply hack into the car’s network (CAN) or direct into the car’s computer and provide the electronic signals. Unfortunately, our BMW was build long before those days so we have to go the mechanical route. The good news is that there really is hope for the day when a standardized interface is available on cars that will allow you to simply plug in your computer and go.
We have found some amazing gadgets from some outstanding vendors. Once the race is over, we will add a vendor list along with details of how we used their products. For now, this is a contest and it isn’t in our advantage to share our research with our competitors. Check back after the race.