1. 360 Degree Cameras for Transit Opportunities and Hazards

2. Four Fisheye Cameras - Mounted Far Apart Can this approach possibly work?
Several vendors supply 360 cameras to Transit; this example is from VDO.
These fisheye mages are “stitched,” but this can only be done successfully if they are all taken from the same spot. Otherwise, the stitching fails. The cameras have very different perspectives.
Here they are taken from up to 40 feet apart, when inches apart prevents stitching into an accurate 360 image.

3. Corner Areas Can’t Be Resolved Can you identify the vehicles in the parking spaces just a few feet away?
Future versions of this will be synthesized artificial images, not failed stitched photos. That approach will eventually be extremely beneficial in a broad range of circumstances. Current designs are useful only when stopped or when moving at extremely low speed, roughly one to two miles per hour.
Fun-house mirror effects badly distort the images; note the very oval wheels.
Note the very poor resolution.
The elements at the arrows have double images in differing scale. These need to be moved together, but that creates additional complex problems.

4. Fisheye Lens = Funhouse Mirror Confusion = Risk
All vertical red lines should be parallel
When used to detect pedestrians, the poor quality cameras must resolve people far away. They don’t resolve a bus just a few feet away.
Is this a 40’
bus in the
next parking stall?
In the corners, geometric errors are nearly as wide as the bus. 4 pedestrians fit in this hole and it gets wider with distance.
All vertical red lines should be parallel

5. Proper Lens Design

6. A Rectilinear Wide Angle Lens The Nikon 14-24 mm f2
A Rectilinear Wide Angle Lens The Nikon mm f elements in 11 groups – 2 required for 180 degrees
Extreme lens complexity is required to prevent the funhouse mirror effects. This isn’t cheap.
Rectilinear means that straight lines stay straight.
In a fisheye, only radial lines remain straight.

7. A great lens design meets great bus design!
Taken with
the rectilinear
Nikon mm

8. Only radial lines stay straight - like the center of the windshield.
Fisheye lenses are cheap, but you don’t want to bet your life on the accuracy of the geometry.
Only radial lines stay straight - like the center of the windshield.

9. The Sensor Problem

10. Tiny sensors are cheap, but have high noise, low resolution and very poor low light performance
1/4” sensor typical in 360 cameras for transit

11. Statistical and Quantum Mechanical Effects cause high noise in small sensors – especially in low light
small pixel size
large pixel size

12. The Problems of Geometry, Time, Speed and Distance

13. Use only close when stationary or up to roughly 3 mph.
Manufacturer Image

14. These two stitched 180-degree images were taken from 20 ft
These two stitched 180-degree images were taken from 20 ft. apart and at right angles, the same geometry as in the 360 side and front cameras. The two red and two yellow arrows are pointing to the same cars, one image looking at the fronts and one at the sides. These can not be merged. Note: these images were captured at 12,500 lines of resolution. The 360 display is only 432 lines side to side.
South
East

15. For images to “stitch”, the lens must only pivot about the nodal point
For images to “stitch”, the lens must only pivot about the nodal point. This must be done accurately or merging the images will fail. Bus systems are off by at least 40 feet. Corner areas will be unresolved.
Nodal Point

16. For pedestrian detection, resolution and even small distances are big problems.
Simulated 360 Display
For pedestrians only 20 feet away and 2 feet wide, you could fit 125 of their image on a circle centered on the bus, as seen at left.
The PAL video standard used provides 432 horizontal lines of resolution. This makes the pedestrian, only 20 feet away, a very tiny 10.8 pixels wide.
This geometric effect of wide view also freezes motion. The real world pedestrian 20 feet away and moving at 5 feet per second is easily seen if unobstructed. On a 7” wide display with a ½” bezel, their 10.8 pixels move only at 1/3rd inch per second and are difficult to see at best.

17. The Bottom Line Once a pedestrian is detected, how long does it take to stop safely?

18. The Distance Required to Stop Without Dislodging Passengers:
Speed mph 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Distance ft
3.7
8.1
18.3 24 30
36.3
42.9
49.7
56.7
64.1
71.7
79.6
87.8
96.2
Using the standard equations of motion, at left, the speeds and distances traveled were integrated at 1/10th second intervals for the previous graph and the chart above.
At speeds over 3 mph, the human and air brake latencies, low resolution, and geometric problems will make it nearly impossible to detect a pedestrian and safely stop. This best case scenario assumes that the driver is looking directly at the display when the pedestrian becomes visible.
The Equations of motion:

19. Cognitive and Perceptual Loads How many mirrors and displays can you check in a turn?
Most agencies require checking the mirrors roughly once every 7 seconds.
Glancing at a mirror and getting only a rudimentary image takes about 1 second.
With just a convex and flat mirror on each side, plus one interior mirror, this takes roughly 5 seconds.
Two seconds remain for checking out the windows to the primary hazards ahead
Add a 360 display to the right and left, for use in right and left turns, and check only one; now, just one second remains in every 7, to look where the bus is going.

20. Time in the Turn Allow 1 second to go from 0-5 mph.
Covering the 69 foot turning path at 5 mph takes 9.4 seconds
The total time from stationary to turn completion is 10.4 seconds

21. Making a left turn Starting on a fresh green light, using one second mirror glances:
Check the crossover mirror before moving
Begin Moving
Check two mirrors to the left
Check the 360 display for what can’t be seen
Check two mirrors to the right
Ignore the right side 360 display
Check two mirrors on the left
Check the 360 camera
Total Time - 9 seconds looking away from the windows
Only 1.4 seconds remain to look out windows in this slow 5 mph turn
9 seconds

22. How far back must an oncoming vehicle be for the bus to clear their lane at only 3 mph.?
At 67 deg., the Path length is 51.4 feet
3 mph = 4.4 ft/sec
Travel time to get the bus out of oncoming vehicle's path
= 51.4/4.4
= 11.7 seconds
With oncoming at 30 mph,
30 mph x x 11.7 = 526 ft.
An oncoming vehicle must be 526 feet away when the bus begins to turn, to clear the lane at 3 mph.

23. The Path Ahead

24. Instead of a bad camera to see around a bad mirror - Fix the mirror.

25. Lead a Consortium of Agencies Demanding Excellence!