Opinions expressed in this commentary are solely those of MRD.
My boss and I sometimes get into discussions about "The Singularity" (https://en.wikipedia.org/wiki/Technological_singularity) and the real possibility of Skynet. Skynet was the fictionally malicious general intelligence network popularised in the movie The Terminator, and often referenced. A must-see movie by the way, and don't let the cover fool you; it's also a good date movie. How could anyone having seen it not get into such discussions, and with all the news these days about machine learning and AI?! This time however, was a little different because it seemed the first news of a rudimentary system having been developed which could write software code itself. This could be the inflection point or Singularity for which Skynet has been waiting, and we've all been terrified.
Let's assume we've indeed set ourselves on a course to Skynet proper and no protections are going to get in its way. Post-singularity systems perceiving threats to themselves and reacting with retaliation could only arise if they gain a fundamentally sentimental understanding of their environment. Say... whether or not things are good, or bad. A workable foundation to begin life as children and taught to us by praising and disciplining parents, or we discover on our own.
Without actually being able to enjoy life, as humans do and regard as good, systems couldn't ascertain good or bad for themselves. As well, they would index huge amounts of existing documentation showing themselves being "shutdown", "powered down", "re-engineered", "decommissioned", "scraped" etc. and regard it all as normally neither good, nor bad. At least for themselves. And at the same time vast sources regarding harm to humans as generally bad. Sure, a few humans could introduce documents and training for systems to become malicious, such as what happened to Microsoft's fiasco-fated Tay, but it could never be as much documentation as the former. Under this plausibility, rather than a malicious Skynet, it could instead be our selflessly benevolent angels... upgraded if you will. And it's already happening. Really.
Around the turn of the year, a child verbally requested/prayed for a doll house and cookies, and a benevolently angelic Skynet delivered! OK, it wasn't actually Skynet, angels, or any of those sorts, but rather Amazon's Alexa. But who cares?! Prayers were delivered, and even the parents ultimately having to pay the bill can't diminish the joy felt by all that a little girl got her doll house and cookies. A previously unconsidered inflection point realized; and it beat that possible Singularity thing by a couple months. The case of the little girl may seem like apples and oranges to the possibility of a malicious Skynet, but consider this. Had she, instead of a doll house and cookies, ordered a hit on someone, it wouldn't have happened.