What three years of ocean data taught me about trusting AI-built software
Most weather and tide apps give you numbers. Wind 12 mph. High tide at 2:14. Waves 3 to 5 feet. What you do with them is up to you.
TideCall doesn’t do that. You put in a ZIP code, and every live station within 75 miles gets one word for today and each day of the week ahead: Good, Fair or Poor. For a skiff or bay boat, or for a kayak.
That one word changes the job. An app that shows numbers can be read wrong. An app that says “Good” can be wrong. If it tells a fisherman it’s a fine day to take a small boat out while the National Weather Service says the water is too rough for small boats, that’s not the reader’s mistake. It’s mine.
So before TideCall gave a single answer, I had AI check the rule against three years of what actually happened on the water.
The rule looked right
The first version of the rule was one I was already running for five spots near home in Louisiana. Look at the wind. Check whether the Weather Service has a warning out. Make sure the tide is moving. Set a few limits, and score the day.
The code worked. The answers looked reasonable. Down here on the Gulf, they were. I could have shipped it to every coast, and nothing on the screen would have told me otherwise.
But “looks reasonable” tests the screen, not the rule. The only real test was to compare its answers against days we already knew the answer to.
Three years of data, three hours
On November 14 I asked the AI to do exactly that: replay the rule over every day from 2023 to 2025 and check each answer against what really happened. Did the Weather Service have a small craft advisory, or something worse, up for that water? (I’ll call all of those warnings from here on.) And what did the nearest buoy, a floating sensor on the water, measure?
That’s 261 locations, about 262,000 days of readings, and 310,000 marine weather alert records (one per area an alert covered). 3,612 files of raw data, about 1.7 GB. The records came from NOAA’s tide stations, the National Data Buoy Center and Iowa State’s archive of Weather Service marine alerts.
It started at 10:20 a.m. Early on I got impatient and told it to speed up, and NOAA’s servers started turning us away within a minute, so it slowed back down and waited them out. A first report on 72 locations came back in about an hour. The full run finished around 1:40.
A few years ago, that’s weeks of work nobody pays for on a project like this, so the rule ships on instinct.
This time it took three hours and twenty minutes, much of it waiting on NOAA’s servers or on me to say go ahead.

What the data said
The rule broke in three places.
The open ocean looked calm when it wasn’t. On the Gulf, the old rule called a rough day Good about once in a hundred. Then it went west. On the Pacific coast, the rule said Good on 20% of the days a nearby buoy measured rough water for a boat: waves of 6 feet or more, or 20 mph wind. In Hawaii, 25%. A pier’s wind gauge can’t see waves rolling in from a storm a thousand miles away.
The Great Lakes were never Good for a boat. Not once in three years. Part of the rule wanted at least half a foot of tide, and the Great Lakes barely have one. A test that made sense on the coast failed every day on a lake.
One warning hour ruined a whole day. Any warning in daylight made the day Poor, even a short thunderstorm warning issued that afternoon, which no morning forecast could have known about.
It also caught a mistake of mine, the exact one I’d told it to look for. When I asked for the check, I wrote: “If Hawaii is Poor 80% of the time, the limits are wrong.” My planned kayak limits came close: Hawaii would have been Poor for a kayak on nearly two days in three. They were loosened before launch.
The biggest lesson: Weather Service warnings, not wind readings, explain most bad days. Wind gauges on piers read lighter than the open water, and on their own they said Good on 7 to 55% of the days a warning was up, depending on the region.
So the rule changed. It now checks the wave height at a nearby buoy, not just the wind. Windy island coasts like Hawaii and Puerto Rico get their own wind limits. A warning has to last three hours of daylight to set the day. And if the Weather Service’s feed of warnings is down, TideCall gives no answer rather than guess. A quiet feed is different: when it’s up and nothing is posted, the day gets its answer.
With the new limits, a boat gets a Good on a rough day less than 1% of the time in every region, a kayak less than 5%. The Pacific went from one rough day in five to almost never, Hawaii from one in four to none. And not by calling everything Poor: Hawaii is still Good for a boat on 38% of all days, the Pacific on half.
What the AI did, and what I did
I’ve written plenty about AI getting things wrong. This time it did the checking.
The AI pulled the records, matched each location to the closest buoy within about 15 miles, replayed the rule hundreds of thousands of times, and laid out where it broke, with numbers. That’s a job I couldn’t have done in a morning, and probably wouldn’t have done at all.
What it didn’t do was decide the check was needed. Nobody asked for it, and nothing looked wrong. Testing before launch was my call, because an app giving answers carries more weight than one showing numbers.
It also didn’t decide what to do with every result. Alaska came back Poor on more than half of all days, with Nikolski, out in the Aleutians, at 85%, and Valdez, tucked up in Prince William Sound, at 6%. Alaska has a warning up about half the time, so that’s probably real. The analysis flagged 31 locations for a person to review. Some are real weather, like bar harbors such as Crescent City and Westport. Some have the nearest buoy outside a reef or a bay mouth, measuring water the harbor never sees. Telling those apart takes an angler who knows the water.
One honest limit: this was a one-time check, run as if the forecast had been perfect, and the live app runs on forecasts. But the limits live in settings, not code, so when another season of data comes in I can run the check again and retune without rewriting anything.
What this means for your software
The moment your software stops showing numbers and starts giving answers, the bar goes up. Approve this order. Flag this invoice. Route this ticket. Mark this customer at risk. Each one is a small Good, Fair or Poor, and each one can be wrong in a way a person reading a report never would be.
Most businesses already have their version of the buoy: a record of what actually happened. Last year’s orders. Invoices that turned out to be disputed. Tickets that turned out to be urgent. Customers who actually left.
Before a new rule goes live, replay it against that record. Look hard at every place it would have been wrong. That used to be the expensive step everyone skipped. With AI, it can take a morning.
The AI was fast at building TideCall, and faster at checking it. Asking for the check is the part a person still has to do.