The effectiveness of agri-environment schemes depends on scheme type, taxon and landscape. Here, we show how spatial scale, i.e. studied transect, field or farm level, and controlling for yield loss, can drastically change the evaluation of biodiversity benefits of on-field (organic farming) vs. off-field (flower strips) schemes. We selected ten agricultural landscapes in Central Germany, each with a triplet of winter wheat fields: one organic, one conventional with flower strip, and one conventional without flower strip as a control. We surveyed the abundance of wild bees at field edges for two years. We found that comparing data at the transect level may lead to misleading conclusions, because flower strips, covering only 5% of conventional fields, support fewer bees than large organic fields. However, a 50% cereal yield loss of organic farming can be considered as equivalent to yield levels of 50 ha conventional plus 50 ha flower strip. This would promote 3.5-times more bees than 100 ha organic farming. In conclusion, considering various scales in the evaluation of agri-environment scheme measures is necessary to reach a balanced understanding of their ecological and economic effects and their effectiveness.