Diets used to induce metabolic disease are generally high in fat and refined carbohydrates and importantly, are usually made with refined, purified ingredients. However, researchers will often use a low fat grain-based (GB) diet containing unrefined ingredients as the control diet. Such a comparison between two completely different diet types makes it impossible to draw conclusions regarding the phenotypic differences driven by diet. While many compositional differences can account for this, one major difference that could have the greatest impact between GB and purified diets is the fiber content, both in terms of the level and composition. We will review recent data showing how fiber differences between GB diets and purified diets can significantly influence gut health and microbiota, which itself can affect metabolic disease development. Researchers need to consider the control diet carefully in order to make the best use of precious experimental resources.