This is the OSU submission to the 2018 Fact Extraction and VERification (FEVER) shared task at the EMNLP 2018 conference. The system uses a heuristics-based approach for evidence extraction and a modified version of the inference model by Parikh et al.(2016) for classification. Our process is broken down into three modules: potentially relevant documents are gathered based on key phrases in the claim, then any possible evidence sentences inside those documents are extracted, and finally our classifier discards any evidence deemed irrelevant and uses the remaining to classify the claim's veracity. Our system beats the shared task baseline by 12% and is successful at finding correct evidence (evidence retrieval F1 of 62.5% on the development set).