I have one- and two-year old children. Needless to say, babyproofing is critical in our home. One of the safety precautions we have taken is to purchase outlet covers for all of our unused outlets.
These little plastic pieces fit perfectly into any outlet, covering all three holes and prohibiting little fingers from entering. Because they are plastic, there is no danger of being shocked when you put them in or take them out. They work really well and are impossible for children to remove.
On the downside, they can be a little difficult for adults to remove too. They can also serve as choking hazards for children. Once while vacuuming I put the outlet cover on a chair (I know, bad move) and soon found my toddler sucking on it. So, as long as you are careful not to leave them laying around (I now place them in my pocket while using an outlet) you should have no worries.
These are very inexpensive, and a very good idea to have in all your outlets if you have little ones in your home.