The availability of weather data from remotely sensed Earth observation (EO) data has reduced the cost of including weather variables in econometric models. Weather variables are common instrumental variables used to predict economic outcomes and serve as an input into modelling crop yields for rainfed agriculture. The use of EO data in econometric applications has only recently been met with a critical assessment of the suitability and quality of this data in economics. We quantify the significance and magnitude of the effect of measurement error in EO data in the context of smallholder agricultural productivity. We find that different measurement methods from different EO sources: findings are not robust to the choice of EO dataset and outcomes are not simply affine transformations of one another. This begs caution on the part of researchers using these data and suggests that robustness checks should include testing alternative sources of EO data.