The New York Times crossword puzzle is a daily ritual for many people around the world, and for those who play often enough, it’s clear that the puzzles become more difficult with each passing day of the week. But for the first time, thanks to Redmond, Wash.-based Puzzazz, an analysis of the difficulty of the Times crossword has been completed — validating popular thinking and offering some surprising results.
Roy Leban is founder and CEO of Puzzazz, an 8-year-old puzzle technology company whose apps allow people to buy and solve puzzles digitally. Puzzazz works with the Times to offer its wildly popular crossword for solving through the Puzzazz app at no additional charge to New York Times digital puzzle subscribers.
Here’s the new twist: Puzzazz has come up with a way to calculate a “Difficulty Index” for The New York Times crossword based on anonymized data from its own users for every Times crossword going back to July 1, 2015. Leban described the project in a blog post Monday, explaining that every future Times crossword will have the Difficulty Index added to its Puzzazz leaderboard page automatically “as soon as we have collected enough data to calculate one.”
Leban shared a bit of insight behind his analysis with GeekWire, and we also hopped on the phone with Will Shortz, the renowned crossword puzzle editor for The New York Times, to get his reaction to it all. (For the sake of clarity, as mentioned by Leban in his post, both men use the term “solve” to refer to when a person works on a puzzle, whether or not they finish it and get the correct solution. The term “complete” refers to when a person solves a puzzle, finishes it, and gets the correct solution.)
Leban’s data relied mostly on the amount of time it took solvers to complete a puzzle. The popular understanding is that Monday’s puzzle is the easiest of the week and by Friday and Saturday, the puzzles are the most difficult. Leban said that he was most surprised that his analysis shows Thursday puzzles to be harder than Fridays, but he attributed that to the fact that Thursday puzzles are known for incorporating more tricks.
Other findings, according to Puzzazz:
- More people solve the Monday and Sunday puzzles than any other day of the week, but more solvers complete the Tuesday and Wednesday puzzles.
- The Sunday puzzle is solved by more people than any day except Monday, but, on a percentage basis, fewer people complete it than any other day.
- The degree of difficulty varies the most on Thursday, which is known for its tricky puzzles.
“I’m impressed,” said Shortz, who has been the crossword editor at the Times since 1993 and is just the fourth in the paper’s history. “I think it’s good work. I’m proud of myself for editing and presenting crosswords that generally do increase in difficulty through the week. It’s not a science. It’s an art. If you do the puzzle on a daily basis through the week, the Wednesday may not always be harder than Tuesday, Thursday may not always be harder than Wednesday. But if you do the puzzle long enough, the trend shows that it does get harder as the week goes on.”
While agreeing that Leban’s work “corroborates” what he does, Shortz was intrigued by the data related to Thursday puzzles, and had his own theory.
“The one little surprise to me in Roy’s analysis was that Friday was slightly easier than Thursday, and I pointed out to him that he was measuring difficulty only on one scale — the amount of time it takes a solver to finish a puzzle. And that’s one measure of difficulty,” Shortz said. “A second measure is whether people can do a puzzle at all, and by that measure I think our Friday puzzles are harder than Thursdays. Fewer people can do the Fridays. The Thursdays are tricky and they can take a while to do, but I think more people can finish a Thursday than can finish a Friday. That’s my feeling.”
Shortz said he hears all the time from solvers who can’t get beyond Monday, or can solve up to Wednesday or Thursday but can’t do Friday or Saturday. He thinks people know what they can do and what they can’t.
“You just go as far as you can,” said Shortz, who also serves as the puzzle master on NPR’s “Weekend Edition.” “You go as far as you find the puzzles pleasurable and when they’re no longer fun, you stop at that point in the week.”
But could a high Difficulty Index — alongside fast times registered by puzzle experts on the leaderboard — discourage some people from solving a particular puzzle? Leban said no, likening the situation to people who love playing pickup basketball. They don’t stop playing themselves just because NBA star LeBron James is so much better than them. Shortz was amused by that comparison.
“The comparison to LeBron James is interesting, because if you’re a basketball player, unless you’re one of an elite few, you never get a chance to go up against LeBron,” Shortz said. “Whereas, if you’re a crossword solver, you can solve The New York Times crossword at home along with everyone else and compare your time with the leaderboard, the national champions. So that’s cool.”
In the end, Shortz firmly believes other factors can impact a solver’s ability to complete a puzzle, and that different puzzles test different people’s strengths on different days of the week.
“You read the crossword blogs and someone comes on and says, ‘This puzzle was easier than the normal Wednesday’ and someone says, ‘It was harder than the normal Wednesday,’ and others say, ‘It’s the same as usual.’ And how do you explain that?” Shortz said. “It’s partly that your knowledge is being tested in the puzzle and sometimes you just know stuff that others don’t so the puzzle is easier. And sometimes the puzzle hits your blindspot and it’s harder and other people are right down the middle. And I find that even your mood and the time of day that you do a puzzle can affect your reaction to a puzzle. If your mind is a little sluggish, the puzzle will feel a little harder than it would have been if you had tried it at a different time of day.
“So, there’s really no science to this. It’s varied.”
While his analysis was driven in part by curiosity, Leban still sees a clear benefit to having the Difficulty Index on Puzzazz.
“Our vision is to be the way people solve puzzles on digital devices, so providing information like this makes perfect sense, and the information is on our web site for everyone, even people who still solve on paper,” Leban said. “Like some other things we’ve done, this is a gift to the puzzle world and a way of making everybody’s experiences better.
“At some point, we’ll add Difficulty Indexes to other puzzles as well, but The New York Times was a great place to start because it’s every day, there’s a consistent plan to difficulty, and we had a lot of data available.”