Can't really remove except through JS, but the neat thing about templates is you can include only the js you need at the end of the views... Unless you're building a single page app & in that case you can include w/ libraries like require or just load everything.
If you need to load different CSS, I would probably use a different layout file and define the different CSS includes in each layout file.