b'78CAMERAS AND CLASSIFICATION The test vehicle is crowned by seven cameras, three of which capture the scene unfolding directly in the front of the car (as seen at right), and four processing anything that might approach the vehicle from the left, right, or behind. Here, after stopping at a busy four-way intersection, the vehicle determines it has the right of way and safely enters the intersection. It stops to allow two pedestrians to cross the street. Each actor in this sceneincluding the FedEx truck creeping into the intersection; parked and moving vehicles; a bicycle parked against a street pole; and all nearby pedestriansare detected by the SDS and classified. The color-coded masks match each pixel of the image to an object classification. Orange masks indicate people, blue masks mean vehicles, and pink masks are for bicycles.LIDAR GOES TO WORKThe same scene is captured by the vehicles long-range lidar sensors. The visual waves represent pulses of laser light beamed from sensors housed in the vehicles rooftop sensor pod. Whenever the beams hit an object, they bounce back to the vehicles sensors as unique points of measurement. Cumulatively, these hundreds of thousands of points generate a point cloud, a visual representation of the surface area of all the objects detected. The point cloud can be visualized in a multitude of ways, in this case as color fields that indicate the objects distance from the vehicle. Each lidar sensor collects 10 of these images every second, ensuring that if anything moves suddenly, the vehicle will notice it.OVERLAYING THE MAPIts not enough just to see whats happening around the vehiclethe SDS must also understand the parameters of the area where its operating. For this, it turns to a 3D model of the area, a meticulously constructed map of the city that features everything a driver (or, in this case, the Argo SDS) should know about the street, its infrastructure, and the laws that regulate it. Here, you will find lane markers that indicate all of the legal pathways for driving, street signs that dictate speed limits and rules of the road, and precisely rendered traffic lights that communicate which lanes they control and the yielding relationships of all road users.PREDICTING THE FUTUREOnce the vehicle has perceived its surroundings, it must make predictions about the intentions of all actors on the road. Here, the SDS highlights potential pathways for each of the nearby vehicles, allowing the Argo vehicle to anticipate multiple possible actions so that its not taken by surprise. The blue line is the route that the Argo vehicle intends to take. The multicolored lines indicate likely routes for other vehicles in the scene.FullBook_Mar24.indb 78 4/25/21 6:43 PM'