Complexity and Tractability
In the traveling salesman problem, we are given a set of cities and distance between every pair of cities, the problem requires us to find the shortest possible route that visits every city only once and returns to the starting point. This is a perfect example of an intractable problem. Intractable meaning problems for which there exist no efficient algorithms to solve them.
On the right are 20 randomly generated points from excel, these designate the 20 different locations a messenger pigeon must visit before returning home. For the pigeon to find the fastest route he can use brute force nearest neighbour or a genetic algorithm all of which I will talk about. These types of algorithms all have different methods of finding as close to the optimal route for my messenger pigeon to take. The problem I have with my messenger pigeon is much like the traveling salesman problem (TSP) and the cray pot problem. This being that the only known way to get the optimal route takes an extraordinarily long time. All these problems are known as intractable problems.
Brute Force
What is tractability, intractable vs tractable. An algorithm is a set of instructions that solves a problem. Some problems appear to have no algorithmic solution other than brute force, and there are so many possibilities that it takes far too long to find the optimal solution. These problems are known as NP-complete (can’t be solved in polynomial time – the time required to solve the problem using any currently known algorithm increases rapidly as the size of the problem grows). The traveling salesman problem is a perfect example of this as the amount of points grows the possible routes increase by a factorial. A factorial is the number of cities multiplied by one less than itself over and over e.g. 5x4x3x2x1 = 5! (! Is the symbol used for factorial). As seen on the right by adding one city the time taken skyrockets. On the other hand, a tractable problem would be a binary search within sorted data this splits the data and looks to see if the desired value is greater or smaller then splits that side again and again till the point is found. This is tractable because when you double the amount of data you are only adding in one extra calculation. A binary searches cost is O(log n ) thus making it tractable. As seen to the right by adding more data the time is only marginally increased. Brute force is a very common method used in search algorithms it is a linear method in the worst case will have to search through every possible combination. The cost of an algorithm is measured in what we call big O, big O notation is used in Computer Science to describe the performance or complexity of an algorithm. Big O specifically describes the worst-case scenario for a specified algorithm. The big O notation for a linear algorithm is O(n), n being the amount of possibilities. On average we would see a O(n/2) because after many attempts we would see it at the half way point. With our traveling salesman problem or pigeon problem as I am calling it, the amount of possible combinations increases exponentially using a factorial. If our pigeon has to visit 5 locations the number of possible routes is 5! And 8 city’s is 8!, this would be 8x7x6x5x4x3x2x1 this is equal to 40320. If we just adding one location (8! to 9!) the amount of possible combinations is increased by 900% from 40320 to 362880.
As seen to the right I have worked out the amount of time it takes to calculate different factorials. This is based off a single core 1Ghz processor (1 billion operations per second). As shown by tripling the amount of locations from 8 to 24 the amount of combinations is increased by one sextillion %. This makes it impossible to use a brute force approach for anything more than 16! (35 minutes to solve). If my pigeon was to only ever have 14 or 15 stops we could remain using a brute force algorithm but when we gain more customers, this will very quickly become impractical.
Even if we found an algorithm that can solve it faster we will never know if it is the optimal or fastest solution unless we brute force it. So even if another method can solve 14! faster it might be worth while waiting an extra minute and a half for the perfect solution. Thus, meaning we must find a better solution for when we have larger amounts of locations.
Considering it will take 77 years for me to calculate my messenger pigeon route I will also be using an example of 10 locations throughout my investigation. On the right is a brute force approach to the 10 location TSP and I will continue to use this map for the rest of the experiment. I will be able to compare how close my other results are to the optimal result (something I can’t do for the 20-city map). This calculation was done with a python made TSP algorithm. It took noticeably longer than 0.004 seconds (more like 5 seconds) But this is due to a very inefficient algorithm and my laptop not being able to run 1 billion operations per second on python. The length of this map (fitness was 622 units), I will be able to compare this to other maps later in the report.
Greedy Algorithms / Heuristic
Brut forcing an intractable problem is never going to work so we must devise a method to come up with a close to optimal solution that is close enough to the point where the solution is useful. Because of this we could use a greedy algorithm like nearest neighbour. Nearest neighbour requires you to go to the closest possible point each time then returning to the start. I used this method on our 20-city map from the start. The map nearest neighbour gave me is not optimal, but it could be worse. The main problem with this route is the three intersections where two lines cross this causes a lot of time wasted. To solve this, we need to do what’s called a 2-opt this is a swap. This is when we take two lines that crossover and swap them to save time. The second map on the right shows the outcome of our first 2-opt this shows the new fastest route. When we swap two lines there are 2 new routes one being faster because it loops the map together and the other splits the map into 2 routes, so this change can’t be done. On the second 2-opt we use the same method to swap the second set of crossing lines and finally we do our third swap to make a final map. If we were to have two lines crossing over one line then we could try all combinations to get the shortest route, this is know as a 3-opt. This technique (without 2 or 3 opt ) almost always yields a path less than 25% of the shortest possible path (even less with 2,3 opt). This is due to it being able to sometimes miss shorter routes which are easily noticed with human insight thus making it a greedy algorithm. The problem that we are facing is that for these 20 points we will never truly know the fastest path due to it taking an extraordinarily long time to brute force. We have to use algorithms like these to get as fast a route as possible as quick and efficiently as possible.
I got a very similar result for my 10 city nearest neighbour. It looks like an effective path except for the intersection so we will need to do a 2-opt swap, there is only one way where the swap will make the path shorter, so we will swap these lines. As seen on the far right our new nearest neighbour map is very similar to that of the brute approach (optimal route). I feel that this route would work fine and there would be very little time lost but for the time it took me to measure the distances and swap the intersections we would be better off waiting 5 seconds for the perfect route. If I was unable to use an algorithm and had to do it by hand (like a pigeon because they can’t code in python). This method would be great – much more effective than going in a random order or numerical order.
Genetic Algorithms
A Genetic algorithm has given me the most success so far with the least effort out of all solutions I have found for this problem. These algorithms are good at taking huge search spaces and navigating them looking for optimal search combinations of things you might not otherwise find. With our traveling salesman problem, we can use genetic algorithms to find a very close to optimal solution. These algorithms are based off the biological process of evolution. We start with a test group of say 10,000 different routes our pigeon could take, these are now our “organisms” we then look at the distance that they take. The paths are broken into sections or traits. When the two paths “mate” random traits or sections are passed on creating a new. An example of this is genetic crossover over. We take a random section of the route from a parent (In this case parent B) and put that into our offspring. I have chosen the section 351 in the middle (Highlighted yellow). To fill in the blanks we then fill up the rest of the route with the cities from the other parent (Parent A) from left to right whilst skipping the ones already entered to avoid visiting a city twice and missing out on another (Highlighted green). We now have a new route that is different but similar enough to keep successful sections if one of the parents is fast. If we made random changes we you just be generating new routes each time and it would just be an overcomplicated brute force algorithm. Each generation this mating process will commence twice to insure we always remain at our desired population.
We then use mutation to add variation to some of the paths. Mutation is when we take a percentage of routes (e.g 5%) and from those routes pick a random selection of points and switch them on the map this is done to add variation so as we are getting closer to an optimal solution there is still change and the routes are not all identical. Our city order might be 1,2,3,4,5,6,7 and after crossover it would change to 1,2,3,5,4,6,7 for example. We also need to add new routes into the gene pool to mix things up so each generation we would implement say 100 new random.
Once both the mating / crossover and mutation processes have occurred the program will commence a culling where the longest 50% of routes are removed to insure these unsuccessful routes don’t go back into the gene pool. Some genetic algorithms may allow the user to choose the amount of generations that it continues for, but others work on a percentage gain. Say in the first 5 generations the average path length decreased by 5 percent each time the algorithm would continue. Then after 200 generations if there was no decrease in the shortest route ten times in a row the algorithm would stop because it isn’t making improvements. Whilst doing this it will keep a record of the shortest route in case a near optimal route is found but it becomes a victim of mutation and is lost.
The python based genetic algorithm I used allows me to set the number of generations that I wish to go through this directly determines how long the program takes to run. Whilst testing I stopped seeing improvements after 100 generations which took roughly 5 seconds, and this achieved what I believe to be a near optimal route. As seen on the right. This program uses coordinates on a 200×200 grid that I entered, and it returned reordered coordinates that I plotted onto the graph.
I have taken my 10-city map and put it into the genetic algorithm. This also took about 5 seconds to run and returned what looks to be a great route. Because of it only being 10 cities, I was able to brute force it earlier. The brute force length was 622 units and this genetic algorithm returned 629 units. This is a 1.1% difference so if all genetic algorithms were able to be 1.1% longer than brute force I think that is a very acceptable error margin considering the decrease in time taken.
Real world application
Many people are spending money and time to try and solves NP hard and NP complete problems NP standing for non-deterministic polynomial-time. This is done because of the extreme profit potential if you have the solution. This is due to the fact that if you come up with a solution for any of these intractable problems you have the ability to solve all other Np hard problems by converting your problem into the one that has a solution. There is an infinite amount of uses for this solution and holding the key to this would unlock all cryptography and encryption meaning encrypted passwords and data would be completely open. It would also lead to a drastic increase in compression efficiency not to mention the ability to calculate the optimal solution for problems previously “Intractable”. Although a lot of research is going towards this it is widely believed that this is impossible, that (P != NP).
The main problem with computers is that they can only look at two points at a time whereas a human can plan further ahead for more than 10 points. This means computers struggle to calculate anything that doesn’t involve two points and a distance between them. For 15-20 points a person would be able to draw a path very quickly that would be close to optimal, but this may take a long time with large numbers, so it is best left to a computer as it can take many more variables into account. A messenger pigeon would be very unreliable and impractical for delivering parcels, so I need a better solution. A parcel delivery van will work great but there is a small problem. Cars have to follow roads as they cannot travel in a straight line like a pigeon would so there is now a far greater number of roads and routes to calculate. The shortest distance route may not always be the fastest, often the speed limit and traffic congestion will increase travel time meaning there are even more things that have to be considered. This is the problem that company’s like google maps must face as waiting a few minutes for your phone to calculate a route is not practical. An example of this is the United Postal Service (UPS), they have 120 thousand trucks operating worldwide and each of these trucks gets 10.2 Miles per gallon (23.06025 litres per hundred kilometres). The worldwide average diesel price is 1.50NZD/ litre. Assuming each driver works 3 days a week 48 weeks per year and that our prefect algorithm saves them 10km each day. They travel 172800000km less each year. Meaning UPS save almost 60 million on fuel each year. Just by taking a 10km shorter route each day. I am sure over a full day of driving you would be able to decrease travel distance by much more than 10km. However, this algorithm would become extraordinarily complex very quickly as on this large of a scale you would want as much time and distance saving as possible. We could look at speed limits, road works, weather conditions, traffic congestion, fuel consumption e.g. starting and stopping vs travelling at a constant speed, tire wear from making more right turns and higher collision areas. We could justify speeding and the chances of getting a ticket vs the time gained. There will always be more considerations that can be taken into account for time gain.
For 11 or less locations brute force is the optimum way to go but for anything more than that genetic algorithms provide the best solution. This is due to genetic algorithms being a liner relationship to the amount of locations whereas brute fore is factorial. As shown in the table to the right as we increase the amount of cities by 4 the genetic algorithm has a 14% increase in the amount of operations required whereas with brute force the amount of operations increases by 22664313423552714842933646377669885952000000%. It is fair to say that the genetic algorithm is without a doubt the most efficient and should be used whenever a brute force approach isn’t able to be completed within an acceptable time limit. In saying this we will never know if the result received from a genetic algorithm is perfect unless we brute force it and compare but that defeats the purpose of trying to solve this faster. But based on the difference between my brute force and genetic algorithm for 10 cities we can expect a 1% increase in distance for the genetic algorithm compared to brute force. (Optimal)
Intelligent Systems
In the not so distant future Artificial Intelligence will become so powerful that we can mimic the human brain and do everything that a human can do, on a computer. Artificial intelligence can be anything from a self-driving car making risk analysis or a basic auto predict that knows what you like to type thus predicting messages you type on your phone. We could have an AI doctor than can diagnose every known medical condition. These sound great, right? Well what if the AI decided to kill us? The car and doctor are examples of Artificial General Intelligence (AGI), AI that can make decisions and mimic human intelligence, but this is just not possible yet. Our AI is too inefficient, and our computers aren’t powerful enough. To put this into perspective to simulate one second of human neural activity it took the 16th fastest supercomputer 40 minutes. So, it’s going to be a while before you can fit one of these in your car, but it will most certainly happen in the future. However, we do have low level rules-based AI as well as basic neural networks that can be used whenever for a multitude of purposes. One of the most helpful being chat bots. These chat bots used canned responses in an attempt to appear human but only a few have truly passed the renowned Turing test. Alan Turing once said, “A computer would deserve to be called intelligent if it could deceive a human into believing that it was human.” From there came the Turing test where an integrator would have a generic nonspecific conversation with both a human and a chat bot, if they cannot distinguish the human from the bot then it has passed the Turing test. This test addresses the shaky boundary defining what makes a computer program “Intelligent”. To be able to pass this test we need to look at how these chat bots work and at how they use keywords within sentences to relate to what the user is asking if it is unable to do this it breaks. This keyword recognition is the biggest flaw in chat bots today. To truly have a conversation the bot needs to look at the whole sentence and understand it and computer can’t understand things unless we tell it how. The bot needs to understand slang, comprehend full sentences and be able to carry a conversation by looking at previous responses then determining the right answer to carry a conversation.
Turing Test One – I was the Interrogator; highlighted segments are myself
Conversation 1 – Talking to my partner Conversation 2 – Talking to Cleaver bot
Hello.
Hi.
What is your favourite colour?
I would have to say orange. What about yourself?
Mine is blue. I love anything that’s blue.
That’s nice. What kind of foods do you enjoy?
I like pizza.
Oh. Pizza is ok I guess.
Why do you say that?
I have just never been a fan of pizza.
That’s a shame. What do you like then?
There are no foods that I really like. But I do like to play rugby.
I don’t like sports, I have bad knees.
Hello
Hi, what are you doing?
Talking to you!
Obviously…
What have you been doing recently?
Laying in bed. How are you feeling.
I am feeling good thanks for asking.
What is your favourite colour?
I would have to say green. What is your least favourite colour?
I like green as well. Green.
Why do you not like green?
Because it's the colour of life.
What is life.
What you don't have.
Conversation 1 – Me talking to my partner Conversation 2 – Mitsuku talking to my partner
Hello.
Hi there.
What did you do last night?
Not much. How about yourself.
I was working on my complexity and tractability external.
That sounds fun .
It was not, it took me so long to wrap my head around.
Do you understand it now?
Yes. What is your favourite movie?
I like John Wick.
I have heard of that. What do you like about it.
I love all the action! What is your favourite.
I would have to say Finding Nemo.
Ah yes, that is a very sad movie.
I know, I cried when I watched it
I liked the part when Nemo was swimming.
Me too, have you seen the new one?
No, I have not. Hi.
Hi there.
What are you doing?
I am waiting for my Mousebreaker.
What is that?
Mousebreaker is my botmaster. He also looks after a few games websites.
That sounds fun. Do you celebrate Christmas?
Yes, it does sound rather good.
I certainly do. I always enjoy Christmas, people are much friendlier towards me then.
I don't like Christmas.
Many people find Christmas a very lonely time. I get a lot of people talk to me on Christmas Day.
Why is that.
That's what would make sense to me.
What do you mean by that.
"That's what would make sense to me." seems pretty clear.
This is confusing.
What is it that confuses you?
The way you are speaking.
Let's try to analyse it step by step. We were talking about unknown . The last male you were talking about was Mousebreaker. Now are you less confused now?
My Turing test
The purpose of the turning test is to determine whether an AI can pass as being intelligent. To do this we need to make it as fair and as even as possible by taking out distinguishing traits that would give it their identity away. My Turning test consisted of me and one of my classmates sitting in separate rooms having a conversation through a piece of paper. This was because the experiment is about what the chat bot responds with, not how they say it. We had two conversations one after the other where he would either be himself or a chat bot and it was my job to distinguish which conversation was with a chatbot and which was with a person. To keep this experiment fair, we had to control variables that could play a factor into differentiating the human from the bot. We had the conversation over paper because of wait times, I could reply instantly as myself but entering their question into the chat bot would take time giving it away. To make it even more reliable we waited on the human test before handing the paper back under the door so that I could not use the time taken to write the answer to tell them apart. We made it clear that all specific questions were off limits such as where do you live? What is your name? How old are you? This was due to the fact that any answer from any one of these questions would instantly tell the interrogator which one the bot is rendering our experiment useless.
I started off being the interrogator, both times I initiated the conversation. For my first conversation I was very hesitant, it was very human like and I couldn’t decide if it was just a really good chatbot or my partner. The second conversation really gave it away for me. The first sign was when the chatbot asked me what I was doing, this seemed like a very odd question for him to ask me as the obvious answer was “talking to you” which was exactly what I replied. When I was answering his question about my favourite colour I replied with two sentences in one message. This was off putting for the bot and it replied in two sentences where a human would have ignored my first sentence as it was cutting off the conversation. A human would of gone straight to the question on the new topic of what their favourite colour was. On top of this its two replies were the same, yet they were conflicting statements. They replied I like green as well, only to answer my question (what is your least favourite colour) with “green”. At this point I knew without a doubt the second conversation was with a bot.
My partner found himself in a similar situation to I where the first conversation could be a person or a great chat bot. But again he knew straight away when Mitsuku replied “I am waiting for mouse breaker”. Whilst reading over this transcript I noticed that Mitsuku replies with much longer sentences, on top of this whilst writing these messages out I never noticed that Mitsuku broke when over explaining what we were talk about. She said, “We were talking about unknown”. This is obviously a pre-set message with an entry field, but I am surprised that the bot said this considering there was an unknown field.
I think that Mitsuku is a much more intelligent bot than cleaver bot, but it doesn’t try and act like a human. Although both failed the Turing test if they made minor changes to mitsuku to make her a human and not openly act/ say that she’s a bot. If this was done I think it would pass the Turing test just because of its vast amount of knowledge and on topic replies.
Chatbot Techniques
Using remembered “canned” responses.
An excellent example of this was with Mitsuku, her whole last message was canned with two entry fields. “Let's try to analyse it step by step. We were talking about unknown. The last male you were talking about was Mousebreaker. Now are you less confused now?”. These entry fields would have been stored during the conversation as we were speaking. In further testing of mitsuku I observed that whenever the bot recognises the user’s confusion it asks what the user is confused about. Only for it to forget that it has asked me a question and treats my reply to her question as a completely different statement as seen on the right. As opposed to asking me what it is about this conversation that confuses me or explaining the conversation to me like a human would.
Having two sentences in one entry
With Mitsuku whenever you say multiple sentences in one message she responds to each sentence separately. This is not human like as often the first sentence is a reply to the question asked beforehand and the second sentence is asking another question / changing the topic. As seen on the right Mitsuku had already said hi to me but said it again in response to my Hello message. I said three sentences, she responded in threes sentences where she should have ignored the first two or at the very least the first one.
Giving vague or general replies
When a bot can’t understand what it is being ask it will either reply with generalised answer related to the question or try and change the topic to make it seem more intelligent. In some cases, it might simply say “I don’t know” but this will make it seem less intelligent, so the bot will try to avoid this answer. As shown in this part of another cleaver bot conversation when asked, “Do you have any family?”. The response was love related because the bot would have learned that the two are related so to avoid the family question it will make a statement about who he loves. By giving a vague reply it can make the user think it changed the topic because it was going nowhere but it was really due to cleaver bot not knowing the answer.
Answering a question with a question
Countering a question with another question or changing the topic are very common techniques that chat bots uses when they don’t know the answer to a question or when they are just trying to change the topic. By asking me a new question they are distracting me from the fact that they avoided my question. Both examples I got from cleaver bot in an extra conversation that I had. In the first one we were talking about movies when I asked what they were trying to imply, the bot got lost and changed the topic to avoid answering the question. In the second conversation when I asked if they wanted a family, they responded with a question about love. This is both a combination of my previous technique (vague and general replies) and answering a question with a question. Cleaver bot picked up on the topic of family but didn’t not know how to answer so responded with a question.
All four of these techniques help to make the bot seem more intelligent. To someone that isn’t intentionally looking for these techniques, they would deceive the user and give them a false view of the bot’s intelligence. But because I knew what I was looking for and what techniques they used I was not convinced by either chat bot that they were a human.
Practical applications of chatbots
Using a chat bot just to have a conversation is very impractical and a waste of the creator’s time. Chatbots can be used for much better things than talking to someone on the internet. The whole idea behind a useful chat bot is to replace a human. For the computer to do a job that hundreds of humans would otherwise be doing. Take a call centre for instance, if you were able to implement a chatbot that everyone has to go through before talking to a human you could cut your staff in half as this bot would be able to solve most basic problems and the more complex ones would be transferred to humans.
Another great use are chatbots for any website where bookings or purchases are made, the chat bot would compare to a salesperson meeting you at the door helping you with your purchase. You ask it how much this is, or where to find this and the bot could directly link you to your desired page. Air New Zealand is a perfect example, you could be greeted by a bot where you enter your flight departure, arrival airport and date all informally to a chatbot. Then the bot gives you all the flight options. This would be perfect and save you a lot of hassle if you could do everything through this bot. Select number of seats, what class of seats, departure times all these could be done by the bot to save time and make the user experience easier. After all, that is the whole purpose of a chat bot is to reduce staff costs and increase customer satisfaction. This would start off with the bot expecting different formats of answers for each question then entering that information into the flight search. As the bot becomes more utilised it could evolve and learn different types of responses for each question. This would be relatively easy for a chat bot due to the bot asking the question and just receiving different forms of answers but in a different scenario where the user would be asking the questions like on a help bot for Microsoft, the bot would be much more advanced as it would have to answer questions and understand the whole sentence that it is being asked whereas a booking bot would look for keywords that the user is telling them.
Evaluation of Turing test
Personally, I don’t think the Turing test is a good method of testing a chat chatbots intelligence like Mr Turing intended. However, I do believe it is a good way to test if a chatbot is able to convincingly behave like a human. We can use methods like making chatbots have bad grammar and use slang to make them more human like. But this is not something that we would define as being intelligent, most chatbots that have passed the Turing test used these methods of deceiving the interrogator as apposed to being intelligent and fully comprehending everything. Because of this I think Alan Turing has incorrectly outlined the purpose of the Turing test by using the word Intelligent. Although this would have been a fair statement in the early 1950’s as the idea of a computer deceiving a human seemed farfetched, yet in today’s world building an AI for this exact purpose would not seem difficult and having this basic ability would certainly not make it intelligent. This is the view that many computer scientists share and express.
For a computer to have true intelligence it would have to show emotion, make decisions and think by itself. It needs to be more than a program that follows rules and looks for keywords to decide which pre-set answer to use. This type of AI is known as Artificial general Intelligence (AGI – mimics human intelligence). To test of a chat bot is at an AGI level it would take much more than one test, it would require countless hours of testing and evaluation and after this we could potentially call a chat bot intelligent. Although we have not currently reached this, we will, without a doubt, in the years to come.
https://www.cleverbot.com/
https://www.pandorabots.com/mitsuku/
http://www.theprojectspot.com/tutorial-post/creating-a-genetic-algorithm-for-beginners/3
https://towardsdatascience.com/introduction-to-genetic-algorithms