Artificial intelligence and machine learning are the latest hot topics that makes everyone super excited in the field of digital transformation. Its applications have found its way to all industries including the use of AI in cyber security.
In this digital era, software development and testing has become more and more complex and most IT companies are facing the challenge of delivering the products on time.
The old school rule-based automation provides a limited scope of reducing human work.
With the combination of artificial intelligence and automation, we can not only make the system efficient and easy but also reduce the human effort. It’s important to understand how this works in the real world.
What is Artificial Intelligence?
Artificial intelligence (AI) is a broad area of computer science which deals with building smart machines capable of achieving tasks that require human intelligence to perform.
Sometimes AI is also called machine intelligence. When machines become intelligent they can understand data and draw conclusions based on it.
Types of AI
AI technologies are categorized into three based on their ability to mimic human behaviour, the technology or hardware they use, their real world application and the theory of mind.
On the basis of these categorizations, all AI systems (real and hypothetical) fall into one of the following types:
Artificial Narrow Intelligence (ANI)
Artificial General Intelligence (AGI)
Artificial Super Intelligence (ASI)
Artificial Narrow Intelligence, also called weak AI, is the only type of AI that exists till today. The other two types of AI are conceptual or hypothetical.
Besides the three categorizations, AI includes machine learning and deep learning as subsets. The image below will give you a better understanding about it.
Machine Learning
Machine learning is the science which allows computers to act without being explicitly programmed.
People who are interested in AI need to gain a better knowledge about machine learning in order to understand how computers learn from data. To break it down, “ML is a subset of AI that trains a machine how to learn”.
How does it learn?
The process of learning starts with data, Whatever the specific task that our model is intended to do, it has to observe the data and look for the patterns in it.
But, how does it see the patterns? algorithms.
ML algorithms are often divided into supervised learning, unsupervised learning and reinforcement learning- which is a broad area that tells us about machine learning techniques and approaches. These algorithms turn a dataset into a model.
Supervised Learning
Majority of practical machine learning uses supervised learning. In supervised learning, the input (X) and output (Y) are labelled, and we can use an algorithm to learn the mapping function from the input to the output.
Classification and regression are the two types of supervised learning techniques.
Unsupervised Learning
This type of learning has only the input variables (X) and no corresponding output variable (Y). This type of learning helps model the underlying structure or distribution in the data in order to learn more about the data.
Unsupervised learning is further grouped into clustering and association.
Reinforcement Learning
Reinforcement learning helps machine learning models make a sequence of decisions. It is done by taking suitable action and earning rewards in an uncertain, potentially complex environment.
It is a more complex and challenging method to understand where machines employ trial and error to come up with a solution to the problem. Two types of reinforcement learning are positive and negative. The two widely used learning models are Markov Decision Process and Q learning.
Deep Learning
A discussion on AI and ML is not complete without mentioning deep learning (DL). Deep Learning is a subset of machine learning inspired from the working of the human brain.
Like how a human brain filters information, deep learning helps to filter the input data through layers of neural networks to predict and classify information.
Deep learning technologies are classified into Convolutional Neural Networks, Recurrent Neural Networks, and Recursive Neural Networks. These are all multi neural network architectures containing a large number of parameters and layers.
What is Web Automation?
Web automation is the concept of letting software programs or robots perform tasks and actions that involve a web browser. Human interaction with computers is repetitive, tedious and causes a number of errors.
With the right automation tool it is possible to automate browser tasks, like scraping data, generating reports, web testing, data extraction etc.
It is a process of automatically performing operations on a web browser to achieve speed and efficiency that wouldn’t be possible with human interaction.
Browsers and websites are used to access information and perform a wide variety of tasks. Getting the final result is one task and the way to get there is another task.
So an automation has to perform everything, starting from opening a browser till it reaches the intended result. Each user or a company will have their own requirement for automating a website, and for that many web automation frameworks are available.
Selenium, Google EarlGrey, Cucumber, Appium, Robot Framework and Gauge are the popular examples. Among these, Selenium is the most widely used automation framework.
Selenium for Web Automation
Selenium can increase code readability which in turn improves test productivity. Primarily it is used for automating web applications for testing purposes, but it is certainly not limited to that.
Selenium can be used to automate all kinds of web based tasks as well. We can use multiple programming languages like Java, Python, Ruby etc to create automation scripts, which makes it more reliable.
Moreover, developers and analysts can utilize Selenium to scrape websites, which is a common method of extracting large amounts of data from websites quickly and efficiently. This demonstrates another practical use of this versatile tool outside of just testing.
Selenium is a suite of softwares and each one fulfills different purposes as needed by an organization. Here is a list of tools:
WebDriver
Selenium Integrated Development Environment (IDE)
Selenium Remote Control (RC)
Selenium Grid
Selenium Remote Control and webdriver are merged into a single framework as selenium webdriver. This tool is used for automating web application testing to verify that it works as expected.
In the current scenario, most people are relying on Selenium and other automation tools, but the time has come for machine learning automation to take over.
One of the disadvantages of using Selenium is that it has dependendency on the web page elements. Even for a click option.
What will happen when the page elements change?
Definitely the test will fail and the programmer has to rewrite the code based on the changes, and it’s very time consuming.
So, this brings us to the important question. How can artificial intelligence, machine learning and deep learning help automate our system?
Using Machine Learning for Web Automation
By using machine learning automation, it can adjust to the changes that happen with time. This makes the test more reliable and maintainable.
For example, AI/ML testing tools can start learning about our application and understand the page without even looking into the HTML script. It can learn about the changes throughout the process.
Once a tool starts learning and observing how applications change, then it can start making decisions of its own at runtime. And if the application keeps changing, it’s no longer a problem because with ML the script can automatically adjust itself.
An example of an AI/ML automation model output is shown below. The model is intended to find the different login types and login forms.
Applications of AI are pointed in three directions namely computer vision, speech recognition and natural language processing (NLP).
Depending upon the need they can be combined or used separately to create a fully automated solution. These are the major components of AI automation.
Computer Vision
Computer vision is a field focused on helping computers to see, which replicates the human vision system. It has a wide variety of applications that we have experienced in many ways without even knowing it.
Some amazing and known examples of computer vision are facial recognition, robots detecting an object and picking it up, medical diagnosis, character reading etc.
The goal of computer vision is to understand the patterns in an image. Which results in creation of a machine learning model.
A model is created by means of training and it requires lots of data. Before training the data needs to be annotated and labelled, that allows the computer to look for patterns in all images that are related to that labes.
Speech Recognition
Like it sounds, speech recognition is the ability of a machine to understand the words spoken. Best examples would be Alexa, Siri, Cortana and Google Assistant.
It is changing the way people interact with their devices. It is simply a digital assistant designed to help people perform a task and access information.
The applications of digital assistance are vast. Some of them are listed below.
Schedule meetings
Make travel arrangements
Print documents on request
Search for information
start a video conference
Speech recognition works using algorithms through acoustic and language modeling.
Acoustic modeling represents the relationship between speech and audio signals and language modeling matches sounds with word sequences to help distinguish between words that sound similar. This made the application of voice search, voice dialing and speech to text processing.
Natural Language Processing
NLP helps computers to understand, manipulate and derive meaning from human language. It basically works with speech and text and its applications are scaling into a lot of industries.
With easy access to the data and computational power, developers can perform tasks such as named entity recognition, translation, sentiment analysis, topic segmentation etc. It allows scientists to achieve meaningful results in the area of media, healthcare, finance and many more.
Don’t get confused with speech recognition and natural language processing. The functions they perform are different from each other.
Speech recognition enables a machine to identify spoken languages and convert it into text. NLP on the other side, understands human language for the purpose of performing useful tasks.
We can get aid from computer vision, NLP, speech recognition, deep learning and related algorithms to achieve the results more quickly.
For testing certain websites or products there won’t be enough test cases available and this makes it difficult for testers and developers. With AI/ML, we can make our automation more stable and reliable.
Why Use Machine Learning for Web Automation?
Reduce Cost
The cost of training a human for a task of testing or doing boring stuff on the web repeatedly is a lost cause. We have to deal with employee turnover and give time for skill development. On the other hand, a machine once trained improves over time and there is no cost involved in repetitive training.
Improve Efficiency
No matter how efficient humans are, they always make mistakes. Automation would be the solution for this problem. A one time learning will be enough for the machine to process the repetitive tasks again and again without an error which in turn increases the efficiency of the system.
Avoid limitation of Manual testing
Running a web application test with over 100 clients is not possible with most software companies, and with a 1000 clients it would be a lot of work for the person who is managing it. And the error it might cause will be large. With AI automation we can save valuable time and make the system efficient for the business.
New Job Opportunities
While AI applications are growing, some analysts predict these efforts will disrupt economic activity in a big way. AI can provide speed, accuracy, computation, etc and the strength of AI with humans can generate better business outcomes.
Conclusion
You can expect a lot of change in machine learning based web automation over the next few years. Being able to train a ML algorithm requires a lot of research and domain knowledge on how a particular system works.
ML algorithms alone cannot achieve the task of web browser automation, but we can make the system more efficient and reliable by working along with traditional web automation frameworks. Even though web automation using machine learning has made huge strides, the best solutions are yet to come.
