Below, Paula Goldman shares five key insights from her new book, Manage the Machine: How to Harness Human-AI Collaboration at Work.
Paula is the first-ever chief ethical and humane use office at Salesforce, where she is a leading figure in implementing AI responsibly. She was also a member of the US National AI Advisory Committee, established by Congress to advise the president, and was named to Fast Company’s AI 20.
What’s the Big Idea?
AI won’t make human judgment obsolete. As the technology becomes more capable, the organizations that benefit most will be those that deliberately preserve human judgment, creativity, relationships, and agency, designing AI around the people who actually do the work.
Listen to the audio version of this Book Bite—read by Paula herself—in the Next Big Idea App, or buy the book.
1. Not every question AI can answer is one it should answer.
When a small business owner reaches out to 1-800-Accountant, a virtual accounting firm, to ask whether their tax return has been filed, the company’s AI agent responds nearly instantaneously. But if the same customer asks a more complicated question, like how to restructure their business to lower future tax bills, they’ll get connected to a human representative.
The distinction is intentional: 1-800-Accountant made a choice about what AI can answer versus what it should answer. Financial advice is heavily regulated, but that’s not the only reason for this kind of pattern. When a customer asks a complex question, it can create an opportunity for a human rep to build trust and grow the relationship.
Getting this wrong can be embarrassing and costly. We’ve all heard the stories: the AI agent that wrongly told landlords they could refuse tenants with housing vouchers or the airline AI agent that hallucinated a policy about bereavement fares and caused a lawsuit. But the equation isn’t just about regulatory or reputational concerns.
Customer preferences and emotions also matter. Research consistently shows that angry customers want to engage with a person. Embarrassed customers want to stick with AI, since it isn’t perceived as judging them. And since acquiring a new customer generally costs a lot more than keeping an existing one, taking these kinds of preferences into account matters a great deal.
2. A little friction is your friend.
In the tech industry, we’re taught that friction is a bad thing. But human judgment and creativity are often essential to achieving good outcomes when working with AI, and small moments of friction can ensure they get applied when they’re needed most.
IKEA’s innovation team experienced this when they set out to break the “bulky, cushion-heavy” sofa archetype, using AI. Standard prompts like “couch” kept reverting to a familiar boxy shape. The breakthrough came when designers stopped to stretch their own thinking about what a couch could be, coming up with capability-based prompts like: platform, lightweight, tent, hammock, or conversation pit.
“Human judgment and creativity are often essential to achieving good outcomes when working with AI.”
The result, “Couch in an Envelope,” was a ten-kilogram aluminum frame with a mesh seat, compact enough to fold in a case and light enough for one person to carry. The prototype was thought-provoking enough to be included in a museum exhibit in Copenhagen. IKEA’s experience illustrates the value of “front-loading the brief.” They pushed people to really think before giving AI a task.
But positive friction can also be useful after the AI runs. You’ve likely experienced the pop-up that asks if you’re sure before you delete a file, or the confirmation screen that makes you verify details before you wire money. Similar techniques are valuable when reviewing AI output for truly consequential decisions, because research shows people tend to trust AI outputs more than the outputs deserve.
Across these examples, the goal is the same: to force a moment where human agency and accountability come to the foreground.
3. AI can reinforce fixed ideas, but it can also help dismantle them.
We hear a lot about the ways AI, used carelessly, can exclude people from jobs, loans, or opportunities they may otherwise be qualified for. When this happens, AI isn’t inventing new rules on its own. It’s magnifying patterns of past human decision-making. But when used deliberately, AI can help disrupt flawed decision-making patterns.
Marc Benioff, Salesforce’s CEO, has talked frequently about bringing AI into Salesforce’s Monday senior staff meetings, where his executives update him on sales progress, region by region. After hearing from the room, he turns to the company’s AI forecasting agent and asks for its read. The AI can sometimes surface problem spots that people in the room don’t want to draw attention to.
A prominent venture capitalist shared a more pointed version of the same idea. Her firm uses an AI knowledge agent that listens to investment committee conversations. If it detects bias in a partner’s observations, the AI flags it. This is an act that might be harder or more awkward for a human colleague to point out, but it allows the team to examine their decision further.
“The AI can sometimes surface problem spots that people in the room don’t want to draw attention to.”
When we think carefully about which assumptions might be outdated, or what valuable perspectives may not be getting airtime, we can leverage AI to help change the status quo. The difference is in the thoughtful judgment and intentionality with which we guide AI.
4. The people doing the work should have a say in how AI influences that work.
On a recent trip to the UK, I met a technology executive who worked with a local town government. This executive (we’ll call him Tom) tried to use AI to help social workers by handling note-taking and paperwork during their field visits. Delegating these tasks to AI even saved the social workers a return trip to the office. The technology worked well, but the humans rejected it, and the program was put on hold.
What Tom hadn’t factored in was that social workers valued the ritual of typing up notes alongside their colleagues, as they brainstormed solutions to harder cases. They also didn’t want to burden their spouses by droning on about the intricate ins and outs of all their clients’ needs. A small tweak—in this instance, understanding the collaborative problem-solving that took place within the team and working that into the design of the workflow—could have made the difference between adoption and an application that didn’t get used.
Listening to the judgment of those who understand the work best doesn’t just help save failed projects—it can drive significant innovation. Amanda Ballantyne, founding director of the AFL-CIO Technology Institute, told me about a long-standing labor-management partnership at a Stellantis Jeep facility to train workers on production robotics. Because instructors were former employees, their solutions came from problems they had actually faced on the shop floor, not from a training manual. Such proximity produced useful solutions that management never thought to ask for. As Ballantyne recalled, one instructor built her own app to troubleshoot a robotic arm: “It was so cool because it gave students a checklist to fix things they would find on the job.”
5. Managing AI is a skill that we need to take more seriously.
We already know what happens when we don’t teach people how to manage well. Plenty of new managers get thrown straight into the deep end, under the false assumption that being a good individual contributor is adequate preparation for managing a team. We shouldn’t make that same mistake with AI.
“Organizations that get transformative outcomes from AI will get there not just on the strength of the technology itself.”
Luckily, learning how to productively leverage AI isn’t as foreign a skill as it can sometimes seem. Aarthi Belani, a partner at Baker McKenzie (a law firm), tells me she sees aspects of onboarding AI as similar to how she would train a junior associate. She gets a good sense of what the technology is good and bad at, what she can and can’t delegate, and expands tasks as results improve.
Aaron Kemmer, the CEO of Magic, a service that provides customers with hybrid virtual assistants (both humans and AI), observes that when customers learn how to delegate well to AI assistants, they actually ask for more help from humans, too. This is because customers have gotten more comfortable with the general skill of how to hand work over productively.
Ultimately the organizations that get transformative outcomes from AI will get there not just on the strength of the technology itself. They’ll get there because they have people at every layer of their organization equipped with the judgment for how to use it well, when not to use it, and when human relationships should still carry the day. Those are skills worth investing in, with at least as much urgency as we’re investing in the tech.
Enjoy our full library of Book Bites—read by the authors!—in the Next Big Idea App:
