A Museum is Using AI to Let Visitors Chat with World War II Survivors

Special to The Washington Post / Katy Reckdahl

17:09 JST, March 31, 2024

Margaret Kerry-Boeke, 94, strode into a gallery at the National WWII Museum in New Orleans. Then she stopped short. Her jaw dropped.

It was as if she were looking in the mirror, at a crystal-clear, almost life-size video of herself.

In the video, Kerry-Boeke sat in an armchair, her legs crossed at the ankles just above her pink ballet-style flats, as she explained her work as a home-front USO dancer during World War II. “I would usually do two tap numbers, and often be called back for a third one,” she said.

Kerry-Boeke is part of what was once called “the greatest generation.” But that generation is vanishing.

Every day, 131 World War II veterans die, the museum estimates. For a new exhibit, called Voices from the Front, the museum filmed the memories of that generation before they’re all gone – and is using artificial intelligence and voice-recognition software to index the memories in a way that will allow visitors to “converse” with World War II-era Americans for decades to come.

Out of the 16.1 million Americans who served in the war, only 119,550 – less than 1 percent – are still living, as of last year’s data from the U.S. Department of Veterans Affairs. By 2036, the department projects, only a few hundred Americans who served in World War II will be alive.

“We’re racing against time,” said Peter Crean, a museum vice president and a retired U.S. Army colonel who spearheaded Voices from the Front, which opened this month.

Of the 18 people interviewed and featured in the exhibit, three died before they could see themselves on screen, including Hershel Woodrow “Woody” Williams, the last living World War II-era recipient of the Medal of Honor, earned in the Battle of Iwo Jima.

“Woody was the last one,” Crean said. “So this is now the only place that you can go to have a conversation with a World War II Medal of Honor recipient.”

Thanks to a $1.5 million donation from a supporter, the museum arranged to make volumetric videos of the exhibit’s subjects using 13 cameras, filming from every angle.

When Kerry-Boeke flew to the film studio in Los Angeles from her home in Leland, N.C., the filmmakers, from a company called StoryFile, asked her to arrive with two identical sets of clothing for a two-day interview.

A large brooch in the shape of Tinker Bell – for whom she served as a Disney model after the war – was pinned in the same spot on her sweater each day. When she sat down in the blue armchair, the film’s production team marked where her feet and her hands were and tracked them all through the shoot, for nearly 1,000 questions. That way, whenever the virtual Kerry-Boeke finishes her answer on video, her starting image moves back to the same position. In between questions, her video face looks ahead intently, as if listening to the visitor’s next query, then nods.

With future technological advances, it’s possible the interviews could become holographs, so that visitors entering the museum might find a three-dimensional Kerry-Boeke sitting in a chair, waiting to speak to them.

The exhibit uses voice recognition to process visitors’ questions and AI to search for relevant answers from the subjects’ interviews. The video responses aren’t themselves generated by AI; instead, the AI finds the best clip from the interviews.

Currently, the AI can sometimes take up to 20 seconds to find an appropriate answer and play the corresponding clip. But as more visitors ask questions, the AI will continue to get “smarter,” improving its indexing and shaving down the response time to a few seconds.

Shortly after the Voices from the Front video went live, Kerry-Boeke watched intently, standing off to the side as her video-self answered a visitor’s question about working on an early TV variety show.

When the answer was complete, Kerry-Boeke’s face lit up. “I was magnificent!” she said, linking arms with Robert Boeke, 98, her husband of four years. She stayed on American soil during the war, while he was part of the 86th Infantry Division, serving in both the European and Pacific theaters.

Two of the people interviewed were home-front workers: Kerry-Boeke and Grace Janota Brown, who made parts for Boeing B-17 heavy bombers. The other 16 were deployed World War II military personnel. They included Theodore Britton, who was among the U.S. Marine Corps’ first class of Black recruits in 1942 and would become U.S. ambassador to Barbados and Grenada; U.S. Army nurse Virginia Leeman Wilterdink; U.S. Army German translator Robert Wolf; Holocaust survivor Ben Lesser; Jeep driver Romay Johnson Davis; and Lawson Iichiro Sakal, from the 442nd Regimental Combat Team, a segregated unit of second-generation Japanese Americans who became the most decorated unit in U.S. Army history.

Kerry-Boeke and her husband first dated in Los Angeles three-quarters of a century ago, several years after the war. Then he graduated from college and moved to San Francisco to work for Mobil Oil; she stayed in Hollywood and spent nine months on a Disney stage in a bathing suit as the reference model for Tinker Bell in the 1953 animated-film version of “Peter Pan.”

That’s been recorded for posterity too.

On the exhibit’s opening day, Crean, the museum vice president, walked up to a Voices From the Front console and scrolled to Margaret Kerry-Boeke’s name on a control panel. Pushing an “ask question” button, he said, “How did you become Tinker Bell?” As the computer’s AI searched her interview for related terms, Kerry-Boeke’s on-screen image appeared to listen and nod. Then it started to tell the story about the fateful day when Disney called, looking for a dancer and actress to serve as a model for a three-and-a-half-inch animated fairy.

As soon as museum officials set out the Voice from the Front consoles, Crean saw families gathering around, with everyone asking questions, sometimes for 30 or 45 minutes at a time. When they left, he said, people would say, “Goodbye!” and wave, as if speaking with a real person.

“They’re talking to a prerecorded interview, listening to answers that were recorded two years ago. But the way the video presents itself allows it to feel real and amazing without feeling phony,” Crean said.

Younger visitors, already accustomed to having long on-screen conversations, interacted easily with the new exhibit, giving curators a preview of how best to engage future generations with the field of history.

To demonstrate the range of questions the video avatars can answer, Crean scrolled to Williams, the Medal of Honor recipient, and asked trivial questions about his favorite color and food. Williams responded immediately.

Then Crean threw him a curveball. “Can you tell me about existentialism?” he asked. “I don’t have an answer to that question,” Williams replied. “Ask me something else.”

Even if some visitors toss out silly inquiries, Kerry-Boeke sees a serious purpose for Voices from the Front. “It can help clear up questions about family members,” she said. “People can ask questions of someone and say, ‘That’s what my grandfather or grandmother did in the war.’”

It was common for family members to hear nothing about World War II from soldiers returning from combat, Kerry-Boeke said. “So many people came home and they would not talk about the war,” she said. “The trauma – that was prevalent.”

A visitor from New Orleans, Devin Dumas, 24, pulled a metal stool up to the console and read through a list of suggested questions for Williams. “What was it like to be a flamethrower operator?” he asked.

Williams, wearing a vintage Marine Corps garrison dress cap and his silver Medal of Honor star on a chain around his neck, explained how, during the fighting on Iwo Jima, he had used his flamethrower to take out Japanese machine-gunners firing from fortified pillboxes. Even though Williams was born 77 years before Dumas and died in 2022, the two men appeared to be having a natural conversation, as Dumas asked about Williams’s childhood in West Virginia and Williams explained how he’d weighed three-and-a-half pounds at birth and wasn’t expected to live.

To Dumas, the experience felt like “sitting in Mr. Woody’s home.”

When Dumas, following the suggested prompts, asked Williams about his best friend, Vernon, Williams’s voice cracked in the video as he recounted how inseparable the two men were, until Vernon was killed on Iwo Jima. Following a promise the two had made, Williams, then 22, slipped a prized ring off his dead friend’s finger and fought the remaining time on the island with Vernon’s ring in his pocket. When he returned home in 1945, Williams borrowed a car and drove the ring to Vernon’s father.

Dumas listened, speechless. The tale reminded him of his bond with his best friend, someone he couldn’t imagine losing, he said afterward.

It’s a reminder that people often connect best to history through other people’s stories – and that what is now at risk of being lost extends beyond veterans’ personal experiences to the memories of many fallen comrades and loved ones, which only live on within this rapidly disappearing generation.