When AI Can’t Replace a Worker, It Watches Them Instead

Whether software that digitizes manual labor makes workers frowny or smiley will come down to how employers choose to use it.


To read Tom Simonite’s original article on Wired, please click here.


When Tony Huffman stepped away from the production line at the Denso auto part factory in Battle Creek, Michigan, to talk with WIRED earlier this month, the workers he supervised were still being watched—but not by a human.

A camera over each station captured workers’ movements as they assembled parts for auto heat-management systems. The video was piped into machine-learning software made by a startup called Drishti, which watched workers’ movements and calculated how long each person took to complete their work.

“In the past, we would take a line that was struggling and bring a bunch of people down with stopwatches to try and make it better,” Huffman says—at least for problems that seemed serious enough to justify the time and expense. Drishti tirelessly logs the “cycle time” for every worker and station all day, for every shift. Plant managers use the data to track output and find and eliminate even subtle bottlenecks in production. “Everything flows better and is smoother,” Huffman says. Denso, one of the world’s largest manufacturers of auto parts, has been testing the technology at its Battle Creek facility since the end of 2017.

Denso’s use of Drishti shows how some jobs will be transformed by artificial intelligence even when they’re unlikely to be eliminated by AI anytime soon. Many jobs in manufacturing require dexterity and resourcefulness, for example, in ways that robots and software still can’t match. But advances in AI and sensors are providing new ways to digitize manual labor. That gives managers new insights—and potentially leverage—on workers.

Some workers say the results are unpleasant. Last year, Amazon warehouse employees in Minnesota staged a walkout to protest how the company uses inventory and worker-tracking technology. They allege that Amazon uses it to enforce a punishing working pace that causes injuries. The company has disputed those claims, saying it coaches employees on how to safely meet quotas.

Workers at Denso were initially wary of the prospect of being video-recorded all day to feed machine-learning algorithms, but Huffman says they have since come to appreciate Drishti’s technology. After something goes wrong, workers can now look at the data and video with their managers, instead of having to hope bosses take their account of what happened seriously. Huffman says having a constant readout on productivity also helps managers be more responsive to nascent problems. “If somebody’s struggling, not every associate is going to call for help,” he says. “If we see their cycle time is jumping through the roof, we can go over and say ‘Are you having any issues?’”

Workers on Denso lines equipped with Drishti’s technology now get a personal feed of their own data. Monitors on each workstation display how a worker is doing, says Raja Shembekar, a Denso vice president. If the worker completes their assembly step on time, they see a smiley face—if not, a frowny one.

Whether AI that digitizes manual labor makes workers frowny or smiley will come down to how employers choose to use it.

Denso originated as part of Toyota, which still owns a stake in the company, and like its parent uses the kaizen philosophy of manufacturing, which encourages workers at all levels to participate in improving how a plant operates. That could lead employees to let workers engage with data and insights from AI overseers in ways that improve their jobs as well as productivity, says Susan Helper, an economics professor at Case Western Reserve University who studies manufacturing.

However US plants, even those applying kaizen, don’t have a good track record of allowing workers to have much influence compared with parts of the world with stronger worker protections and culture. “In Japan and most other countries, there would be some collective way the workers could say ‘This is too fast,’” Helper says. “In the US plants there often isn’t.” Denso’s US plants are not unionized, but Shembekar, the vice president, says the company has a good relationship with all its workers.

The difficulty of balancing workplace data collection with employee trust convinced New York startup StrongArm to limit the uses of its own worker-tracking technology, a motion sensor worn on the torso. The device logs how a person bends and twists, data that software converts into a safety score intended to reduce injuries in workers whose jobs involve lifting or moving objects. Toyota tested the technology at a plant in Princeton, Indiana, late last year.

Preventing injuries is valuable to employers, but data from StrongArm’s sensors could conceivably also measure productivity. Although that might increase the technology’s value in the eyes of some customers, cofounder and CTO Mike Kim says his company decided it would undermine workers’ trust in the devices. StrongArm’s contracts state that if a customer uses the data in a punitive way, the company can terminate the deal immediately. “We do not capture or care about productivity at all,” Kim says. “It kills the initiative because you don’t get the buy-in.”

Algorithms can watch and digitize workers’ every move, but squishy things like feelings still matter.