Blackbox Test Failures
Stories of blackbox test failure.
Wrong build with dev branch
A build was released to production which is based on an active dev branch (v2.6+dev) instead of the release branch (v2.6) by mistake.
The dev branch was ahead of the expected release branch for a new feature development.
All blackbox tests passed the build as the new feature development luckily did not break any existing features, and there was no impact on production.
It was not noticed until it caused upgrade failure to the next version 2.8.