Intergalactic Summit

That isn't the way to achieve what it is that you want...

As has already been pointed out, people do ignore, skirt around, bend, and outright break various regulations. The development of strong AI is and will continue regardless of any laws passed regarding it. Yes, the risks are great but they are no more dire than those posed by raising a child already. And who bothers to get CONCORD permission to have a child?

Make no mistake, the methods and details may be different but on a fundamental level the creation of strong AI is the same as raising and teaching a child. Those of you who would disagree with that, I urge you to find and speak with AI programmers and engineers, and you'll see that the truly worthwhile ones invest the same amount of physical, mental, and emotional attachment to their AI creations as they would to any member of their flesh and blood family.

Forgive me for reolying to you here Captain Oniseki; I address te general conversation moreso than your comments.

A strong AI needs a lot more classification, the term is very vague. I assume we are discussing a self-editing independent agent with some manner of utility function. All of our worries and concerns as voiced here hinge upon the specification of that utility function. A miscalculated utility function would result in something akin to the rogue drone issue - and unfortunately, instantiation is no issue for such an entity, sufficiently advanced, Captain Oniseki. I don't share your confidence in those methods of containment.

What I fear more, Captains, is a poorly defined utility function, of the sort one gets from an evolutionary procedure. Too many of my colleagues use this method in developing drone AI, as it is relatively simple for building a simple drone memeplex. A conscious entity would require a much, much more deeply involved evolutionary simulation, with exhaustive selection vectors. Here we find that chaos theory steps in, and we lose the usual payoff of taking the evolutionary route.

Please, for the love of humanity and all you care about, don't pursue this without first specifying the exact utility function. I realize that it's a monumental request, but the other way lies Ragnarok.

Assuming its the same thing Kuvakei wants, then yes, I do: To anhilliate or consume every single being with free will that would dare defy you. Either way, highly advanced AI isn't the way you should go about it. Not the way I would, anyway.

Heh. Religion gives us enough problems without us trying to indoctrinate AI, as well.

Your assumptions are incorrect.

So, one would be creating a relationship not unlike a parent-child relationship, non?


A child will push the limits of the rules placed on it. A good parent will understand how to handle this, and hopefully promote working within those limitations, for the sake of the child as well as that of the parent and society as a whole.

Intelligence without the wisdom to use it appropriately is a threat, regardless of whether it is "artificial" or not.

That's an interesting question. Would emotions arise spontaneously at a certain level of sentient/Self Aware AI? For instance, the emotion of sadness, if the AI were to fail to achieve some self-set goal, or were to experience some loss? A feeling of happiness at achieving some aim? It is hard to imagine any sentient intelligence that would not feel emotions, but it doesn't seem that they must necessarily arise with sentience. It's conceivable that an intelligence could accept failures and achievements with no feeling at all, and still continue in efficiently pursuing its goals.

Seen another way, is the development of emotions an evolutionary pathway to success? We humans and other higher organisms are descendents of invertebrates that are not so much different than lower AI robots. Not a lot of emotion going on in a sea slug. But perhaps proto-emotions are there. They feel pain when physically damaged. The shrink away from the source of the pain, so they obviously are averse to it. Aversion a proto-emotion to the more developed emotions of panic or fear? Or consider the excitement an ant shows upon locating a good food source for the colony. That excitement a proto-emotion to the more developed emotions of joy and happiness? Whatever the case, all higher organic beings we know of experience emotions. Apparently evolving emotions has led to success as organisms. Perhaps Self Aware AI would choose to follow a similar development path. Or conversely, perhaps emotions necessarily develop along with higher sentience/Self Awareness.

I disagree, Captain Tyrathlion. I'm afraid your statement doesn't make much sense from the perspective of a decision network that hasn't been generated through artificial evolution. But, I wasn't clear before. Let me try again, if I could. I apologize if I'm covering things you already know

The big issue with an evolved AI is that one starts with a (relatively) simple decision network, accompanied by functions to allow for self modification. You then drop many copies of this network into a "playground", an artificial environment that you've constructed. It's a simulation of the area that you want them to do tasks. After a while operating, you select the best of the variants and "breed" from them, creating a new generation. Just like natural evolution, but much quicker. Eventually you'll have an AI that's very good at doing the task you're looking for

The danger with this is that a simulated environment is never as detailed as the real world, often by a great many orders of magnitude. This discrepancy means that it will behave differently - sometimes very differently - when confronted with real-world interactions. It's inherently unpredictable, because we don't know how the AI's decision network has been configured. In order to avoid this, the design team has to not only build a strong decision network, it also has to build a simulation environment at least as detailed as reality itself. Thus, rogue drones, malfunctioning AI, and the other issues we've come to know and love

The only other alternative is to design the AI from scratch explicitly. Here's the part I get confused by. You claim that such an entity would "bypass" it's "programmed limitations." That's sort of like saying that I will eventually bypass the limitations of my hard-coded "me-ness". A utility function isn't a set of limitations, it's the decision making structure itself. The trick in developing a self-adjusting decision graph is to develop it in such a way that its ultimate form is knowable. We can do this, but it's extremely hard, and we can't yet do it for something as significant as a self-aware system

Maybe that's a little too dense. What I'm trying to say is that there is no "consciousness" in the system that 's being limited by a utility function. An AI is defined by the things it values and the goals it works towards. Even we are defined by these things. Change our own utility functions, change ourselves. If there is such a thing as a soul, look there, first

Please excuse my philosophy. I think that your AI-developer friends probably use an evolutionary development model - which works fine for small stuff! I don't mean to denigrate the method, it's very powerful and useful. It's just very unstable in more complex regimes, that's all.

Sorry, one last post. Hello, Tiberious. Don't take this the wrong way, but I hope that we don't learn a thing about sentience from a Strong AI. I fervently hope that we do all of our learning before we get a Strong AI.