What is a robots.txt file for websites?

I am looking into SEO for my website and found something called a robots.txt file for spiders and bots. I guess it tells search engines how to cache your site? Anyone have a good explanation? And in my text file why would I disallow the robots? I'm just a little confused. thanks for your help