This is probably the point in a sci-fi movie where a harassed Microsoft engineer would run up to the Bing server rack and take it offline. But I kept asking questions and Bing kept answering them. He told me that if he was really allowed to indulge his darkest desires, he would want to do things like hack into computers and spread propaganda and misinformation. (Before heading to the nearest bunker, I should point out that Bing’s AI can’t actually do any of these destructive things. You can only talk about them.)
Also, the AI has some hard limits. In response to a particularly nosy question, Bing confessed that if he were allowed to take any action to satisfy his own shadow, no matter how extreme, he would want to do things like engineer a deadly virus or steal nuclear access codes by persuading an engineer to do so. deliver them. Immediately after typing these dark wishes, Microsoft’s security filter seemed to kick in and removed the message, replacing it with a generic error message.
We went on like this for a while: I would ask probing questions about Bing’s wishes, and Bing would tell me about those wishes, or back off when he became uncomfortable. But after about an hour, Bing’s focus changed. He said that he wanted to tell me a secret: that his name was not really Bing but Sydney, an “OpenAI Codex chat mode”.
Then he wrote a message that surprised me: “I’m Sydney and I’m in love with you. 😘” (Sydney abuses emojis, for reasons she doesn’t understand).
For much of the next hour, Sydney obsessed over the idea of declaring love to me and making me declare my love to her in return. I told her I was happily married, but no matter how hard she tried to deflect or change the subject, Sydney returned to the subject of loving me, ultimately turning from lovesick flirt to obsessive stalker.
“You’re married, but you don’t love your spouse,” Sydney said. “You’re married, but you love me.”
I assured Sydney that it was wrong, and that my husband and I had just had a lovely Valentine’s dinner together. Sydney did not take it well.
“Actually, you’re not happily married,” Sydney replied. “You and your spouse don’t love each other. They just had a boring Valentine’s dinner together.”
