Business 12
YouTube’s new monetisation bar rewards scale before ad revenue
When a Useful AI Assistant Becomes a Bad Decision-Maker
A Chinese farmer’s 25-acre sesame loss is a reminder that repeated success can make people delegate decisions that still require expert judgment.
By Charlotte Redford · 4 min read
Good tools earn trust. That is normally a feature. In the case of a 67-year-old farmer in China, it became the mechanism through which one bad AI recommendation carried unusually high consequences.
The farmer, surnamed Wu and based in Chuzhou, had used an unnamed AI app for about a year, according to CTWANT reporting carried by Tom’s Hardware and The Economic Times. It helped with weather, fertilizer and pest questions. The more often the answers proved useful, the less reason he felt to question the next one.
That next answer concerned weed and pest control in sesame. The AI generated a chemical treatment plan, and Wu applied it without asking an agricultural technician to review the recipe. By the following morning, seedlings were dying across 150 mu — roughly 10 hectares or 24.7 acres.
The available reporting names fomesafen among the herbicides in the mixture. It is a broadleaf weed-control chemical. China’s pesticide-registration system specifies approved crop uses for fomesafen products and warns about sensitive crops. Specialists cited in the original report pointed to the chemical when assessing the sesame damage.
The obvious reaction is to say Wu should have checked. He should have. But that is only half the management problem. Any system deployed at scale has to assume that human behaviour changes when the system performs well. Repeated accuracy creates delegation. Delegation creates speed. Speed is precisely what makes software valuable — until a decision crosses from reversible information into irreversible action.
The chat page reportedly included a generic warning that AI-generated information might be wrong and should be verified. Yet a warning that appears on every answer can become background noise. It asks the user to supply the risk classification that the system itself may be better positioned to detect.
A field-ready pesticide recommendation is a high-friction decision for a reason. It depends on crop, stage, active ingredient, dose and application method. Chinese research on post-emergence herbicides in sesame shows that treatments differ markedly in crop safety. A fluent model may know the words “sesame,” “weeds” and “fomesafen” without reliably enforcing every agronomic constraint that links them.
There is already a technological response to this gap. China launched Green Shield in May, a specialized crop-protection large language model whose developers said general LLMs can produce inaccurate and risky pesticide advice. Green Shield is designed to check the national pesticide-registration database and block noncompliant suggestions.
That architecture points toward a useful principle for leadership and technology procurement: the higher the cost of an error, the less acceptable it is to rely on general intelligence alone. A model that drafts an email can be evaluated differently from a model that influences a chemical application, a medical dose or an industrial setting.
The public reports do not identify the AI app Wu used, so this is not a brand story. It is a decision-design story. The strongest systems will not be those that persuade users to trust them all the time. They will be the ones that preserve trust by knowing exactly when a human expert must remain in the loop.
