The capacitor is like a tiny battery. Once it's fully charged, it can unload enough current to light a LED for just a few seconds… or even milliseconds.
Capacitors come in two forms: Electrolytics (left) or non-electrolytics (right).
Electrolytic capacitors are much larger and look like small cans. They hold a much greater capacitance. They are polarised – meaning the long leg must be connected to the (+) and the short one to the (-). Be careful to hook them the right way or they can blow up!
Non-electrolytic (or ceramic) capacitors look like tiny capsules and are non-polarised (like resistors). They can often hold a very small capacitance (reserve of energy). It's usually in the range of 1pF (picofarad) to 0.47µF (microfarad).