To keep things simple I mentally bucket things into different timescales:
- Immediate: less than 60 seconds.
- Bathroom break: less than 5 minutes.
- Lunch break: less than 1 hour.
- Overnight: less than 12 hours.
Here are some tricks I use to avoid waiting.
Use Less DataThis is a simple strategy which people nonetheless fail to exploit. When you are experimenting or debugging you don't know enough about your data or software to justify computing over all of it. As Eddie Izzard says, ``scale it down a bit!''
Sublinear DebuggingThe idea here is to output enough intermediate information as a calculation is progressing to determine before it finishes whether you've injected a major defect or a significant improvement. Online learning is especially amenable to this as it makes relatively steady progress and provides instantaneous loss information, but other techniques can be adapted to do something like this. Learning curves do occasionally cross, but sublinear debugging is fabulous for immediately recognizing that you've fat-fingered something.
The clever terminology is courtesy of John Langford.