What is the difference between a dataset and a statistic?

According to "The Oxford English Dictionary" a dataset (or data set) is: "A collection of related sets of information that is composed of separate elements but can be manipulated as a unit by a computer." This information is presented in the final form and has already been manipulated to produce a set. If you need to look at a relationship between multiple variables across various conditions, then you would need a dataset.

A Statistic is: "The collection of numerical data on a particular subject."  A statistic or group of statistics are typically presented as a finite number such as a percentage or rate and are represented visually often by charts and graphs. 

You will need to think about what type of information you need when you're searching for data. For example, do you just need one statistic about a specific population? Or do you need to see how an organization has manipulated their data to show changes over time?

Many datasets can be downloaded into a statistical package such as SPSS, STATA, and/or R or other types of applications such as MS Excel.  When downloading these files, be sure to look for the accompanying metadata files that are sometimes called ReadMe, Codebooks, or Data dictionaries.  These additional files may be needed to open the datasets, as well as understand what is in them.  The important key factors, such as when and how the data was captured, along with explanations of the abbreviated variables names will help you make use of the data.  These files may even come in handy during the citation process.