Research

With the parents getting older, the cars getting larger and fancier, and the garage not getting any bigger, we embark on a small quest to determine if we can solve self-driving—from the outside. Elon clearly has it wrong. Cameras on the cars!? Silly. What you need are sensors placed seemingly randomly on walls telling the car what to do. Obviously.

Many new cars do have the ability to self-park, but most (especially these two cars) are not precise enough for what I am after. One car will need exactly 4 inches of room to its right, no more, no less. The safety systems on the cars don’t care for that kind of intimacy. If we can’t trust the cars to do it themselves, what might work instead? It’s not as though this is not a solved problem around the world. Robotics is hot again, and there is no shortage of ideas implemented for everything from humanoids to robodogs. Cameras with ML, ultrasonics, lasers, radar, feel—you get the idea. And they run the gamut from super cheap to expensive, new or used. And unless you haven’t experienced the joy of computer component pricing lately, let me paint you an incredibly expensive picture. So I can sell it to you and pay for all these incredibly expensive parts.

So I want to be cheap, but precise. Simple but robust. Usable day and night at the same time (dark garage, bright sidewalk) and rain or shine. Lidar became the front runner. But that was only part of the equation.

A solution to track both cars in the moment and over time and predict current and future motion is only useful if the driver can see and make corrections based on the output of that math. The “simple” answer was an LED sign at the end of the garage. You open the garage, the sign turns on, and it has a giant arrow pointing left or right or center. As you drive into the garage, it tells you to turn left, turn right, or go straight until you are at the exact same parking spot every time.

So a lidar and an LED. Great, now it needs to connect to something. Luckily, the house is already a homelab. Plenty of compute/VM/network/power to spare. Only needed to upgrade Proxmox on the test N100 mini PC, then upgrade all the VMs and LXCs for the new Proxmox version, then build out a new isolated VLAN / firewall / hijack routes on the UniFi system, then run cabling and wire ports and switches for that VLAN. Easy.

The gamble was the lidar. Could we use a lower-cost lidar and get the results we were looking for? The Unitree L2 had the right distance and shape, but the tradeoffs Unitree made to keep it cheap could be an issue. That is, on paper it looks like it has the accuracy, but it’s the details that matter. How many returns in how much time over how much of a spread would determine if this would work or not. Minus that whole “cars are complex shapes plus materials and paint” thing, but that’s another day’s problem.

Digital Twin

As a sensible person with access to Claude and Codex, I did what I think most would do—spent a week on an entire synthetic digital twin of the garage/cars/outside world with real-world physics and exacting timing—to “de-risk” the purchase of the L2. To see if the laser return pattern and timing made sense for our use case. Annnnnnd it kind of worked. You start to make assumptions on assumptions and try to predict every little thing eventually, and it makes less and less sense. What I landed on was this: based on the math, at 10 m away from the front of the garage door, we might have 17 “dots” of connection on a car in a 50 ms window.

(Right, we didn’t talk about the accuracy/time/latency thing yet. Essentially, we can trade time for accuracy. We can tune the system to wait 50, 100, 300, or 500 ms to “paint” a target before sending that data to estimate the tracking path. Longer wait, more dots. However, we are not tracking a static target, but a MOVING target. Moving at what speed, you ask? Anywhere from 0.1 mph to 4 mph. From 20 m away to 1 m. The car does not stop moving while you are doing your collection. Depending on how fast you set your collection and how fast the car is moving, you could miss anywhere from an inch to several feet between laser passes. The L2 is a spinning device. It has a front side and a back side for data—kind of like a lighthouse. The lighthouse will show light on the front and back. As it spins, it sweeps across: you light up the item for a bit, then it’s dark, then you light it up again on the back, then it’s dark, and so on.)

Then we add latency—real-world latency—on top. Latency from L2 to car and back, L2 to computer, computer to LED sign, LED sign to human brain, human brain to steering and pedal changes. Rinse and repeat. Lottttaaaa maths. Most of those delays were negligible. The human latency and the L2-to-car-to-computer latency were the biggest—generally speaking.

The evidence suggested we should make sure we had a 30-day return window. It might work with all the flaws of the L2 and tradeoffs to accommodate, but there are a lot of variables involved. So we bought everything and started over.

Reality

Let’s jump ahead a little for the photo below. What you are seeing is a screenshot from the live tool showing the returns for each of the two sides of the L2 at the 50 ms setting (the pink boxes will come up later, but they were about X, Y, and Z correction). This is inside the garage with two cars. With the top of the L2 facing directly towards the garage door. For the L2, the most dots were at the top. Then, as you get further away, they start to fan out, which I think you can see represented in the photo. It was all about tradeoffs. How do we get the most dots in the least amount of time, both far away and close?

We get the L2, we get the homelab set up, and we plug everything in. The L2 turns on. Womp womp womp womp womp womp. Hmmmm, is that normal? Will that be foreshadowing?

Because of how the L2 is designed—a “feature” of it—the returns from the front and back are not identical. We initially thought, well, perhaps that might be a benefit due to the limited number of returns we will see at the far end of the driveway, where we need the most help. Turns out, no. And more expensive lidars have this solved. But we were not trying to spend thousands of dollars on a science experiment. Further foreshadowing.

Longer story compressed, we picked the wrong tool for the job. The L2 simply did not have the amount of data and precision at the speed and distance we needed. So it was returned.

What follows is everything after we decided it was not going to work, to see, well, what if we tried… just to see if we could make it work.

Negative

Dots. How do we know what a car is and is not? Let’s make a positive and a negative. Take the cars out, scan the room in detail. Anything that gets added after “should” only be the cars (minus the whole “the L2 drifts a bit” thing). So that’s what we did. Made negatives for the garage empty, the garage empty with the door open, and many variations. The photo below is at the 500 ms setting, so it is easy to understand for humans visually. Seems like cars! But as you saw before, that isn’t what the system sees. It might catch just a few dots in the 50 ms window.

Zooming out, if we open the garage (still at 500 ms), we can see the driveway, the street, and wayyyy in the back, the neighbor’s fence 39.84 m / 130.7 ft away. And I know that because, when we first started to build out the digital twin, Codex and I used a laser rangefinder to build an exacting map of distances from where the L2 would be placed. This will be the second major hurdle to the system.

Problems

Time. Distance. Accuracy. We had issues with all 3. First out the gate, the VM’s internal clock was different than what the L2 data was reporting. It was half as fast. It’s a documented issue with the L2. Codex made a local patch, and timing issues were fixed. HOWEVER. The time I did not account for was how long it takes to test things.

The L2 was spinning for hours. It gets hot. Turns off. Gets cold. I was taking measurements through the entire temperature envelope of a garage as well as the L2. It took a while to find root cause, but we discovered that the L2 was reporting data differently in X, Y, and Z depending on how long the L2 had been on. That was in addition to a baseline delta being wrong compared with what we “know” the actual distance from the L2 to be. And it was not a linear distance issue. The data was all over the place. Codex took quite a bit of time to try to build the most accurate and cleanest patch to fix the math.

Tolerances were probably not built for what I was asking for. Which isn’t super great for a tool designed to say, “Hey, here is your car, EXACTLY. You need to move 5 inches forward and 3 inches to the right. Exactly.” That is how you end up with something like the photo below. Building digital boxes at different X, Y, and Z coordinates to collect information about different solutions and how they impact the accuracy over time at different distances for X, Y, and Z. Probably better to see it in 3D video, but we are keeping it 2D for the ol’ blog.

Target discrimination kept being an issue due to the dots not landing at the same spot every time. Despite making a “negative,” we kept seeing the floor outside the garage. Below are different views of what the L2 sees when the car is out of the garage, to help diagnose whether we could improve accuracy at the latency we were asking for.

Tools

And we didn’t even get to implement the full cool LED sign from Adafruit! Ultimately, the L2 didn’t deliver on what the ask was. When asked for time, it was wrong. Distance? It was wrong. And the quoted 2 cm accuracy? In our setup, not even close. And that’s without even getting to moving cars. That was just static data. Wrong tool for the job.

As a learning exercise, super interesting. As a solution, a failure.

In the end, Elon might have been right about using built-in vision systems. Current v2 system running right now in the garage? A hanging tennis ball.

🎾