If you use VLC to watch the video lectures you download, one of the things you should try is speeding up playback (dig around the menus of VLC to find out how). Before chipmunk speed you reach a point where everything happens a bit faster than is natural, but remains comprehensible; only, you have to stay correspondingly alert to keep up. I doubt if the tiny bit of time it saves is worth anything, but the little bit of pressure you put on yourself to stay focused can help a bit.
(And after all, you can still pause, go back and say "say that again?" just like you do when watching at normal speed).