Yes ADSL is terrible, HD videos solely will make you lag; multiple devices and small activities will add up and also make you lag. Turn off wireless internet and devices when not in use. Also you'll lag when downloading regardless if it uses enough bandwidth so you're wrong in that regard.
You have to think about it as a two laned road, if there's a lot of small cars then it's going to be slower likewise if there's one or two large trucks or buses it'll slow down as well.
2 months ago