To show a striking example of a glitch in the puzzle solution algorithm:
http://en.lichess.org/training/22849Rb8 on move 2 failed for me but it's mate in 3 just like the solution.
Are you sure there's a mate in three? I see
1. Qc7 Ke6
2. Rb8 f4
3. Ke4 Nf5
4. Rb6 Nd6
5. Rd6#
Compared with the solution which goes:
1. Qc7 Ke6
2. Rf7 f4
3. Ke4 f3
4. Qe7#
Unless I'm overlooking something with your proposed line, what I said is how Stockfish plays it. But for it to be considered a fail instead of just a "good move, but try again" seems harsh.
No you're right, i somehow got tricked by the online engine. I still want to understand the reason for a 'try again' though I've played mate in 3 when there's mate in 2 and get 'try again' so what's the problem here?
It's based on the engine evaluation at the time the puzzle was created I believe. If the evaluation drops by enough points on a move to be considered an inaccuracy, mistake, blunder, etc. then you fail. If it is a small enough drop in evaluation to not be annotated, then you get the "good move, but you can do better" dialog. At least, that's what it seems to be the case to me from looking at these for the last few weeks. It's also maybe why on rare occasion now the engine will choose the "good move, but you can do better" option if you play from position rather than the puzzle's solution, because the engine had an upgrade recently so maybe some of the puzzle solutions generated are now number 2 lines rather than number 1 lines. I think that's going to be fairly rare though.
(Which is of course the problem with the mating puzzles, because it used to be that any time you went for a longer mate, you got an inaccuracy.)