The availability of weather data from remotely sensed Earth Observation (EO) data has reduced the cost to economists of including weather variables in econometric models. Weather variables are common instrumental variables used to predict socioeconomic outcomes and serve as an input into modeling crop productivity in rainfed agriculture. The use of EO data in econometric applications has only recently been met with a critical assessment of the suitability and quality of this data in economics. We document variability in estimates of agricultural productivity in six countries in Sub-Saharan Africa using nine different EO data sources. By varying the source of the EO data we demonstrate the magnitude and significance of measurement error. We find that estimates are not robust to the choice of EO data and outcomes are not simply affine transformations of one another. This begs caution on the part of researchers using these data and suggests that robustness checks should include testing alternative sources of EO data.