How to maneuver an automatic car

FAQ

A German version of the FAQ is available below.

1. What is AutoNOMOS Labs?

AutoNOMOS Labs is a project of the Artificial Intelligence group of the Informatics Institute (Dahlem Center for Intelligent Systems) at the Freie Universität Berlin. The researchers develop autonomous and driver-assistance systems with the aim of preventing future road collisions. Through the application of modern software and sensor technology, road safety can be improved. The 3 labs Computer Vision, 3D-Perception and Cognitive Navigation are working together on ideas that will define mobility in the future. The concern was supported by the Ministry for Education and Research as part of the ForMaT program. ForMaT supports promising early-stage research activities with preparation for knowledge and technology transfer.

2. What is MadeInGermany?

The basis of the test rig “Made in Germany” is a modified VW Passat, with a specialized drive-by-wire system, and fitted with sensors for environmental perception and positioning. Its “brain” is a software platform, which processes and analyzes sensor data, observes rules and makes decisions for the navigation and behavior of the vehicle.

3. How does the computer pass commands to the vehicle?

MIG is equipped with a special interface for drive-by-wire. That means that a computer (in our case, a laptop) can send direct commands to the steering, and to the brake and gas pedals. Our laptop is connected to the so-called CAN bus of the Passat. A CAN bus is a standardized electronic interface for automotive applications, and different components of the vehicle communicate over it. In our case, we use the CAN bus to send computer commands directly to the breaking, accelerating and steering actuators. For steering, an additional motor is fitted to the steering column. In this way the system can accelerate, brake and steer around corners; it also has access to functions like the indicators and the horn. For this we developed specialized software (the AutoNOMOS system), allowing the car to drive safely and smoothly from one part of the city to another, without need for intervention from the safety driver.

4. How does the car know its location in the city?

MadeInGermany is equipped with a highly precise GPS system. A roof antenna receives signals from the GPS satellites, with which a GPS-CPU calculates the position of the car on the earth’s surface. In cases of lost signal reception (e.g. in tunnels), an inertial unit takes over the work, using accelerometers and an electronic gyroscope to update the vehicle’s position on the map. With additional correction signals, the GPS unit provides vehicle location at precision of less than a meter. Thus, MadeInGermany always knows what street and which lane it's on.

5. How does the car recognize obstacles on the street?

Made in Germany is equipped with 6 LUX laser scanners from IBEO, three fitted in the front of the chassis and three in the rear. Each emits light pulses across an angular field of view. Based on the time interval between emission and registering of an echo, the relative position of every obstacle on the street can be inferred. With 6 laser scanners, the test rig has a 360 degree view of all obstacles on or near the street. The scanners are reliable to a range of 100m. At intersections, vehicles from both left and right are recognized in good time.
Additionally, MadeInGermany has a roof-mounted rotating laser scanner with 64 separate rays. The frequency of the light is infrared, making it invisible to humans. The energy of the lasers is low, so that it is in no way dangerous for human eyes. If human eyes could see infra red, the scanners would appear as twinkling light sources. The roof-mounted rotating laser scanner has a view range of 70m, so that we have two redundant systems for recognition of pedestrians and vehicles.

6. Does the car also recognize pedestrians?

Laser scanners allow the detection of vehicles with high precision, so that we can calculate distances, directions and relative speeds. Pedestrians can also be detected. The laser units are so precise that even leg movements can be processed. For safety reasons, the car stops whenever a static obstacle stands before MIG - whether a vehicle or a pedestrian.

7. Are traffic rules observed?

Traffic rules are anchored in the software. The maximum speed on every stretch of road is recorded in the street map. It's not necessary then to read traffic signs with cameras and image processing - this information is already available electronically. At intersections, other vehicles are recognized and right-of-way rules are observed. The car drives centrally in its lane, and adjusts its speed to always maintain an adequate braking distance to vehicles in front. Observation of fundamental traffic rules (junction behavior, priority from the rights, etc), was already demonstrated in March 2011, with simulated test scenarios at the former Tempelhof airport.

8. Does it recognize traffic lights?

MadeInGermany is equipped with three video cameras fitted behind the windscreen. The central black / white camera perceives the street and provides the position of lane parkings. This allows small errors in GPS data to be corrected, and lets the car position itself centrally in its lane. We use a commercial lane departure warning system from our partners at Hella Aglaia GmbH.
For traffic light recognition, we use the other two (color) cameras on the left and right sides of the windscreen. Traffic light positions are recorded in the street map, so that the car “knows” which intersections have lights, and where these lights are placed. From the color images, the computer recognizes the characteristic form of a traffic light, and the color to which it is switched. Thus, the car perceives traffic light states promptly and at safe distances. Deactivated lights - those not displaying green, amber or red - are a bigger challenge. In demonstration on 17th September 2012, the car correctly recognized all 46 lights in sequence. This is with ordinary and unmodified lights.

9. For how long has the car been tested in road traffic?

Since 2007, autonomous tests have been conducted on the site of the former Tempelhof airport. There, we could extensively simulate and test all routine traffic situations. We were able to demonstrate the safe observation of traffic rules, and that even when unsafe commands were given over computer, the car suppressed these. As part of facilitating this, we designed a circuit to monitor and constrain the vehicle dynamics (e.g. steering and vehicle speed), and installed this in the trunk.
In Spring 2011, after years of functionality and safety tests and in collaboration with TÜV NORD, we developed a comprehensive safety concept for testing on public roads (see FAQ 10). The application for a test-driving license was thoroughly evaluated by LABO Berlin, and permission for public driving granted in June 2011. We were ready to start right away.

10. Is it dangerous?

A condition of the TÜV license is - as with all previous autonomous vehicles worldwide - that a safety driver must be behind the wheel and and able to intervene at all times. While the car drives on Berlin’s streets, the safety driver and a co-pilot monitor all activity and the surrounding traffic. The computer reports its next action with a synthetic voice system (e.g. “traffic light - red - recognized”). In this way, the safety driver knows that the software is running correctly. If the safety driver does want to take control at any time, a light touch of the brake pedal is sufficient. On the dashboard and in the foot cavity there are additional emergency buttons, that can shut down the computer in less than a second, and return full control to the safety driver. As an additional safety measure, there is a remote control which can - with the press of a single button - externally force the vehicle to an emergency stop. Hence, additional safety personnel can stop the vehicle from outside.

11. Is the car insured?

It's insured against 3rd party damages of up to 100 million euros. All autonomous test drives are reported to the insurer in advance. After a long search, HDI Gerling Industrial Insurance proved open to innovative research, and now supports our concern with the slogan “Insuring the future”. Without the free provision of this specially designed insurance solution, our achievements on public roads wouldn’t have been possible.

12. How fast can MadeInGermany drive?

Since the car received permission for road driving, we've made a number of test drives in Berlin’s traffic and on the autobahn. On the autobahn (Avus) it's already been tested past 100km / h (62 mph). The performance is smooth and safe. In street traffic we conform to the normal road speeds (ranging 30 - 50km / h, or 20 - 30 mph).

13. How many autonomous cars have you developed?

Our first autonomous car was called Spirit of Berlin, and participated in the November 2007 DARPA Urban Challenge. After just 9 months development time, and with a relatively small budget, Spirit of Berlin progressed to the semi-final, ranking in the top 30 teams worldwide.
In November 2009, with support from the Ministry of Education and Research, project AutoNOMOS began. Under this framework, two further test vehicles are being developed. With the autonomous car Made In Germany we are - among other activities - fusing data from sensors of many multiple manufacturers, providing new focus for industrial developers. In November 2011 we began working with an electric car, unlocking entirely new application areas for autonomous vehicles. Electric vehicles in combination with intelligent systems will play an important role for the future of mobile societies.

14. Can these cars already be deployed today?

The technology developed in this project can already be used on private land. It would be possible to design flexible autonomous vehicles for use in, for example, airports or for material transport in factories. The greatest problem is the adaptation of the legal framework for road traffic, along with insurance questions. For this reason, the technology is exclusively applicable on private land.
The next step could be use on freeways. Freeways in themselves are closed areas, with simple and easily maintainable traffic rules. Autonomous driving and coordinated driving of vehicles on freeways could be conceivable within a few years, given requisite legal developments. The last step would be autonomous cars on the street, but this will still require some decades of development, and some decades until the concept receives popular acceptance.
The achievements of the research laboratory consist in most part of small but important insights in relation to sensor technology, and methods such as mathematical processes for improving driver assistance systems. A glance at the market for driver assistance systems shows that most market-ready systems come from this research area. We are in regular exchange with automotive manufacturers and component suppliers, that see the long term potential to profit from our results.

15. Who finances the project?

AutoNOMOS was funded by the Ministry for Education and Research, through the German High Tech-Strategy. In November 2009, the concept of our research laboratory won funding under the ForMaT program, in a competitive bidding process. The objective of ForMaT projects is to develop future technologies and bring them to a commercially competitive stage. The German economy is particularly dependent on innovation in road vehicle technology. The Free University of Berlin had already supported the project from 2007, and made the necessary infrastructure available. Alongside research activities and as part of our teaching commitment, students are educated in the fields of software development for robotics and automotive applications.

16. Are we standing before a transport revolution?

Autonomous driving will rather be evolutionary, and introduced alongside a gradual adjustment of traffic law. The driver assistance systems of today already offer aspects of autonomous driving, e.g. automatic parking and automatic speed control. Further assistance systems are continually being introduced at the luxury end of the market. Some vehicles offer pre-crash systems to prepare for emergency braking, or even collision avoidance systems which initiate emergency stops. Many component suppliers work on further systems which will continually find entrance to the consumer market. In this way, the required technology will sit one day in every vehicle. The time to allow full autonomous driving will be a question for government.

17. You describe your car as "green". Why?

In our vision of the future, autonomous vehicles will be deployed as “taxis” in urban areas. Many people today neglect the advantages of car sharing, due to the dominating inconvenience: it's a pain not to have a car available on demand at any time of the day. One solution would be cars that independently picks passengers up. With a signal to the coordination system, the next available vehicle collects the passenger meter-precise from their current GPS position, and proceeds directly to the desired destination or connection to other mode of transport.
This would allow an optimal mix of cars, buses and other transport modes to be achieved. Through the transport of multiple passengers per vehicle, road traffic density could be reduced. Cars, currently spending 80% of their time unused, would no longer pose a burden. Remaining cars could flow faster and more efficiently through traffic. Time and energy would be saved.
This is a vision of the future which is only viable in the long term, and requires social acceptance. It is however an attractive alternative to the current situations in metropolises like Tokyo and Mexico City. In the states with high populations only beginning to increase personal mobility - China and India - this offers an opportunity to pursue a more environmentally sustainable course. This is why we describe autonomous driving as "green".

18. How many people work on the project?

AutoNOMOS is led by Professor Raúl Rojas; Tinosch Ganjineh is technical director; and Patrick Vogel is the financial director, responsible for securing early commercial connections. Twelve full time researchers, six students and further external researchers work on the project. In recent years, many students have also written their degree theses with AutoNOMOS. There are strong connections to other research groups in Germany and the USA.

19. What’s different from Google’s autonomous car project?

Nothing! We share the autonomous driving vision of the Google team. Members of the Google project come from Stanford and Carnegie Mellon Universities. We know one another and swap ideas.


1. What are the AutoNOMOS Labs?

The AutoNOMOS innovation laboratory is a project of the Free University of Berlin at the Professorship for Artificial Intelligence (Prof. Dr. Raúl Rojas, Dahlem Center for Intelligent Systems) of the Institute for Computer Science. The scientists are working on the development of autonomy and driver assistance systems with the aim of avoiding future traffic accidents and increasing road safety through the use of modern computer and sensor technology. In the three laboratories Computer Vision, 3D Environmental Assessment and Cognitive Navigation, ideas for shaping the mobility of the future are jointly developed. The project was funded by the Federal Ministry of Education and Research as part of the ForMaT funding program (research for the market in a team). ForMaT supports promising research approaches already in the early phases in the preparation of the knowledge and technology transfer.

2. What is MadeInGermany?

The basis of the test carrier called MadeInGermany (MIG) is a standard VW Passat Variant, which has been converted with a special drive-by-wire system and special sensors for detecting the surroundings (laser, radar and cameras) and for positioning (GPS). The “brain” of the moving robot is software that evaluates the data from the sensors, observes rules and makes decisions about the navigation and behavior of the vehicle.

3. How does the computer send commands to the car?

MIG was equipped with special interfaces for drive-by-wire. This means that a computer (in our case a laptop) can send commands directly to the brake and accelerator pedals, as well as to the steering. The computer is connected to the Passat's CAN bus via a special interface. The CAN bus is a standardized electronic interface for the automotive sector. Various components of the car communicate via the CAN bus. In our case, we use the CAN bus to transmit the control commands from the computer directly to the actuators for braking, accelerating and steering. An additional electric motor was attached to the steering column for steering. In this way, the system can accelerate, brake and steer around bends and also has access to functions such as indicators and horn. For this we have written special software (the AutoNOMOS system) that enables the car to drive safely and elegantly from one point in the city to another without the safety driver having to intervene.

4. How does the car know its position in the city?

MadeInGermany is equipped with a highly accurate GPS system. The signals from the GPS satellites orbiting the earth are received via an antenna on the roof. The GPS central unit uses these signals to determine the position of the vehicle on the earth's surface. If the satellite signals fail (for example in a tunnel), an inertial unit takes over the task of updating the position of the vehicle on the map using special acceleration sensors and an electronic gyroscope. With additional correction signals, the GPS unit can calculate the position of the vehicle with an error of less than one meter. MadeInGermany therefore always knows which streets and which lanes it is on.

5. How does the car recognize obstacles on the road?

MadeInGermany has six LUX laser scanners from IBEO.Three of the scanners are installed in the front of the body, the other three in the rear. Each scanner regularly sends a light pulse. Based on the reflection of the pulse and the time until the registered echo, the distance to every obstacle hit on the road can be determined. With the six laser scanners, the test carrier has an all-round view (i.e. front, back and sides) of all objects that are on or next to the road. The scanners have a range of over 100 meters. Cars approaching from the left or right can be recognized in good time at intersections.
In addition, MadeInGermany has a rotating laser scanner that throws 64 laser beams into the environment. The frequency of the light is in the infrared range, which is why the light pulses are invisible to humans. The energy of the light pulses is low enough that none of these scanners are dangerous to the eyes. If we could see infrared rays with our eyes, we would notice the scanners as a flashing light source. The rotating laser scanner on the roof of the vehicle has a range of 70 meters, so we have two redundant systems for recognizing cars and passers-by.

6. Does the car also recognize passers-by?

Laser scanners are sensors that are able to detect cars with great accuracy, so that we can calculate their distance to our car and even their relative speed. Passers-by can also be detected with the laser scanner. The laser units are so precise that we can even record and process leg movements. Whenever a static obstacle - be it a car or a passer-by - is in front of MadeInGermany, the car stops for safety reasons.

7. Are the traffic rules observed?

The traffic rules are anchored in the software. The maximum speed allowed on each street is shown on the map of the city. It is therefore not necessary to record the traffic signs with video cameras - this information is already available electronically. At intersections, our vehicle recognizes approaching cars and observes the right of way rules. The car drives in the middle of its lane and adapts its speed to that of the cars in front so that a safe distance is always maintained. The car can briefly stop and continue at the intersections and at stop signs. Compliance with the basic traffic rules (crossing behavior, right to left, etc.) was already demonstrated in March 2011 in special test scenarios at Tempelhof Airport.

8. Are traffic lights also recognized?

MadeInGermany also has three video cameras mounted inside the vehicle behind the windshield. The middle black and white camera captures the image of the road and calculates the position of the lane markings. This can be used to correct slight GPS errors by centering the car in the lane. We use a commercial "lane departure warning" system from one of our cooperation partners, Hella Aglaia GmbH. To detect traffic lights, we use two color cameras that are mounted on the left and right behind the windshield. Traffic lights are entered on the road map so that the car “knows” at which intersections a traffic light signal can be expected. With the help of the color cameras, the computer recognizes the special shape of the traffic light and the color of the signal that is switched on. In this way, the car can recognize the status of the traffic light in good time at a safe distance. Turned off traffic lights that show neither red, yellow nor green are a greater challenge. At a demonstration on September 17, 2012 in Berlin traffic, we showed how the car recognizes 46 traffic lights in a row. These are normal, unmodified traffic lights.

9. Since when has the car been tested on the road?

Tests have been carried out in autonomous operation on the site of the former Tempelhof Airport since 2007. There we were able to extensively test all common traffic situations. We have shown that the vehicle obeys the traffic rules and made sure that even if you force an incorrect command via the computer, the car “refuses” to carry out unsafe commands. For this we have installed an electronic circuit in the trunk that monitors the dynamics of the vehicle and z. B. simply prohibits large steering movements at high speeds.
After many years of functional and safety tests, a detailed safety concept for operation on public roads was developed in the spring of 2011 in cooperation with TÜV NORD (see point 10). The application for an exemption for test drives was conscientiously checked by LABO Berlin and the permit for the State of Berlin was granted in June 2011.

10. Is that dangerous?

One of the conditions for the TÜV report and the exemption for test drives is - as has been the case with all autonomous cars worldwide - that a safety driver (similar to a driving instructor) has to be ready to react behind the wheel. When the car is driving on the streets of Berlin, this person, together with a co-pilot, monitors the activity of the vehicle and the road traffic. The computer reports the next actions via a synthetic voice (such as "traffic light red" recognized). This way the safety driver knows that the software is running properly. If the safety driver wants to take control, a light touch of the brake pedal is enough. Emergency switches are also installed on the dashboard and in the foot area so that the automatic system can also be "thrown off" in a matter of seconds and the driver has full control. As an additional safety measure, we have a remote control that can stop the car with a single push of a button. An additional security person can stop the car from the outside.

11. Is the car insured?

Our car is insured against third party liability for up to 100 million euros. All dates of the autonomous test drives are reported to the insurance company in advance. After a long search, HDI Gerling Industrie Versicherungen has shown itself to be open to innovative research projects and is now supporting our project with the slogan "INSURING THE FUTURE". Without the free provision of this very specially designed insurance solution, the road traffic project would not have been feasible.

12. At what speed can MadeInGermany drive?

Since the car was approved for road traffic, we have already completed test drives in Berlin city traffic and on the autobahn. On the Autobahn (Avus) we have already driven at up to 100 kilometers per hour. The car drives smoothly and safely. In road traffic we drive at the usual 30 or 50 kilometers per hour.

13. How many autonomous vehicles have you developed?

Our first autonomous vehicle is called "Spirit of Berlin" and was registered in November 2007 for the international robotics vehicle competition (DARPA Urban Challenge) in California. After only nine months of development and with a comparably low budget, we managed to advance to the semi-finals and thus to the top 30 teams worldwide.
In November 2009 we started the AutoNOMOS project with the support of the Federal Ministry of Education and Research. Two test vehicles were developed as part of the work. The autonomous vehicle MadeInGermany, on which, among other things, the fusion of sensor data from different manufacturers is tested in order to set new impulses for the industry. Furthermore, an electric car was presented in November 2011 that opens up completely new areas of application for autonomous functions. Electrically powered cars in connection with intelligent systems will play an important role in the mobile society in the future.

14. Can these vehicles already be used today?

The technology developed in the project can already be used on private premises today. It would be conceivable e.g. B. to automate airports or material transport in the factory with the help of flexible autonomous vehicles. The biggest problems are the adjustment of the legal framework for road traffic as well as questions of insurance law. This is why the technology can only be used on private property today.
The next step could be use on the motorway. The motorway is in itself a closed area, where the traffic rules are simpler and can be easily observed. Autonomous driving on the autobahn and coordinated driving of vehicles would be conceivable in a few years, depending on the development of the legal situation. The last step would of course be the use on the road, but for that we need a few more decades of development and a few more decades until there is sufficient acceptance in society.
The research results of the innovation laboratory largely consist of small but important findings in dealing with sensor technology and methods as well as mathematical processes for improving driver assistance systems. A look at the market for advanced driver assistance systems shows that many of the systems that are ready for the market today come from this research area (see point 16). We are in regular contact with manufacturers and suppliers in the automotive industry, who also benefit from our results in the long term.

15. Who is funding the project?

The AutoNOMOS project was funded by the Federal Ministry of Education and Research as part of the German high-tech strategy. In November 2009, the concept for setting up an innovation laboratory was selected from numerous applicants in the new federal states for funding in the ForMaT program. ForMaT projects are about developing the technologies of the future and making them usable for business. Germany as a business location is particularly dependent on new ideas and innovations in the automotive industry. Freie Universität Berlin has supported the project since 2007 and provided the necessary infrastructure. In addition to the research tasks, students are also trained in software development in the robotics and automotive industries as part of our teaching assignment.

16. So are we facing a revolution in road traffic?

Rather, autonomous driving will take place through a slow revolution and slow adaptation of traffic rules and automobiles. Driver assistance systems already offer partial aspects of autonomous driving, such as automatic parking or automatic speed control. Further driver assistance systems will gradually be introduced in the premium segment. Some cars already offer “pre-crash systems” that prepare the car for emergency braking or, in the event of danger, do emergency braking itself. Many supplier companies are working on additional systems, which will then gradually find their way into conventional vehicles. One day the technology will be available in every vehicle. When the time comes is decided by the legislature.

17. You describe your vehicle as a "green car". Why?

In our vision of the city of the future, autonomous vehicles will be used as autonomous taxis in metropolitan areas in a few decades. Many people do not use the advantages of car sharing today because it is too inconvenient not to have the vehicle readily available at all times. One solution would be for these cars to pick up the passenger themselves. A signal to the switching system is enough and the nearest vehicle picks up the passenger at their current GPS position, accurate to the meter, and takes them to any desired point in the city, such as the next connection point for other means of local and long-distance transport. In this way, an optimal mix of automobiles, buses and other public transport options could be achieved. Transporting more than one passenger per car could reduce traffic density. Cars that park unused 80 percent of the time are no longer a burden. The few remaining cars could flow faster and more efficiently in traffic. This saves time and energy. This is a vision of the future that can only become a reality in the long term and that requires social acceptance. In the long term, however, there may be a real alternative to today's situation in cities like Tokyo and Mexico City. For countries with a high population like India or China, which are only just beginning to increase the individual degree of mobility, there is an opportunity to take a more environmentally friendly path. That is why autonomous vehicles are “green vehicles” for us.

18. How many people are working on the project?

The AutoNOMOS project is led by computer scientist Prof. Dr. Raúl Rojas, technical director of the project is the computer scientist Tinosch Ganjineh. Patrick Vogel, who has a degree in business administration, is the commercial manager who ensures early access to the market. Twelve academic staff, six students and other external doctoral students are working on the project. In recent years, many students have written their theses on topics related to autonomous driving. There are research links with other universities in Germany and the USA that are working on the topic.

19. What is the difference to Google's “autonomous car project”?

There is no! We share the vision of Google employees to enable autonomous driving. The project staff come from Stanford and Carnegie Mellon Universities. We know each other and exchange ideas.