Big data and the delayed flight

This is the time of the year when a few long-held beliefs of childhood get exposed (wink wink) and now today comes another: the pilot of your delayed flight really can't make up time in the air to offset the delay on the ground.

So claims Boston Globe data analyst Evan Horowitz and he's got science on his side, he says.

Kristopher Karnauskas, a climate scientist at the Woods Hole Oceanographic Institution, pored over the data from hundreds of flights, finding that what we're told isn't really true that often.

Some flights really do make up significant ground. Looking just at flights that were delayed 30-90 minutes, Karnauskas found that about a third of them actually made up a full 15 minutes. In a separate analysis, the data journalism site FiveThirtyEight.com found that planes sometimes speed up when the delay is small enough and there’s a chance of arriving on time.

Before you get your hopes up, though, there are a few things to keep in mind. First, this isn’t unique to delayed flights. A fair number of flights that leave on time also gain 15 or 30 minutes in the air —so you can’t attribute all of this to “making up time.”

Also, while it’s true that some flights do see big gains, others actually fall further behind. The chart below gives a sense for how balanced this really is. The tallest bar shows the most likely outcome for delayed flights: They make up 5 minutes. And while those flights at the far right of the graph can make up a full hour, those at the far left lose nearly as much.

Horowitz' advice might sound familiar, too: Sit back, relax, enjoy the flight, and plan on getting where you're going a little late.

That's what faces anyone heading to the northeast today. Ground stops because of weather are in effect at some of the New York-area airports and when New York isn't happy, the rest of the country ends up suffering.

Does it have to be this way?

Some University of Michigan researchers are working on a solution, Phys.Org writes this week.

"Airlines generally deal with weather delays after they happen," (researcher Amy) Cohn said. "We want to give them the ability to be a bit more proactive. When they're able to predict delays further in advance, they'll be able to do a much better job of communicating with passengers and optimizing resources."

Cohn said the data from the project may be used to build computer-modeling software that could predict the outcome of an infinite number of hypothetical flight and weather scenarios, helping airlines spot likely weather delays in advance.

That knowledge could enable airlines to adjust their schedules to account for weather patterns. It may also lead to new options for passengers. For example, airlines could look several steps ahead to predict a future flight delay, then offer passengers a pre-emptive rebooking to avoid it.

"Imagine you're scheduled to fly out of Detroit four hours from now and there's a storm in Atlanta." Cohn said. "The airline could use this data to determine that the storm in Atlanta is likely to delay your plane. They could then contact you and offer you a seat on an alternate flight. You save time, and the airline doesn't have to accommodate you on a later flight after the delay happens."

Check it out.

Flight Aware's misery map also shows the current situation. Denver is also a mess. Minneapolis St. Paul is relatively OK, where only six currrent flights are delayed.

• The future of NewsCut? We need your thoughts
• Good night and good news
• Storytime with Bob: A treat outside of the blog
• Favorite NewsCuts? Here are mine.