Creating voice applications is getting easier every day with platforms from Amazon, Google, and Microsoft, and testing and development tools from third parties like Bespoken. And thanks to the constant improvements in voice platforms such as Alexa or Google Assistant, it is possible to create more complex and richer voice experiences. This allows developers to offer easy-to-use and highly useful voice apps to the growing number of users who regularly interact with businesses via voice.
Given all this, it’s no surprise the number of voice applications are expanding rapidly, as can be seen in the following graphic for Alexa skills
But how many of these skills are error-free and offer truly delightful experiences to customers, experiences that bring them back again and again?
Top 4 Most Popular Bot Design Articles:
We have been digging (*) and found some interesting information:
More than 65% of the most popular skills have a rating lower than 4 stars. Undoubtedly, the reasons for having a low rating are varied and go from an insufficiently attractive content to errors in the app code or platform AI components (i.e. speech recognition and NLU issues).
Whatever the reason, a user who does not have a positive experience when using a voice application may give your skill a negative review and almost certainly will not return again. To avoid user churn, it is necessary to guarantee that in addition to providing high-quality content, the application is free from errors. And testing is the only way to ensure this.
For many working on voice experiences, testing is time-consuming, tedious, and error-prone. At Bespoken, we decided to change that. Using the Bespoken tools is an easy solution to guarantee that your voice app is always performing at its best, without taking too much time or money. Let’s take a look.
The 4 layers of testing
These pieces together help assist with every aspect of testing for voice, in an automated way. We cover the entire development lifecycle, so whether you are a developer, QA person, or product manager, we have you covered and we tools that can help you do your job better and in turn, ensure your users are consistently delighted.
A kind of testing done during the coding phase. It ensures the code is working correctly in isolation. As it is executed locally, there is no need to deploy your changes to the cloud each time you update the code. Needless to say, it executes in a matter of seconds.
How to get started?
- Install the Bespoken CLI and use the proxy command to execute your voice app code locally. It is also possible to debug the code with your favorite IDE to quickly find errors without time-consuming deployments to the cloud.
- Start creating simple yet powerful unit test scripts with Bespoken’s YAML based syntax
- Check this sample project to see a real-life example of unit testing an Alexa skill.
This type of test is executed when the code has been completed to ensure the entire system is working correctly — AI (ASR + NLU), code, and external services. The utterances defined on these tests interact with the real Alexa or Google services.
How to get started?
- Use Bespoken’s YAML-based syntax to create easy to understand and maintain end-to-end test scripts.
- Use Bespoken’s CLI’s to execute the end-to-end scripts. Check this video to know how.
- See a full example of end-to-end testing for a skill here.
Continuous Testing (Monitoring)
This type of test ensures that a voice app, once deployed, works flawlessly by verifying your voice app on a regular interval.
How to get started?
- Sign up to Bespoken Dashboard and get a Virtual Device to start interacting with your voice app programmatically.
- Create a monitoring script in less than 5 minutes. The script should contain a set of interactions that test the most important functionalities of your app.
- Enable monitoring for the script you just created. We’ll execute the script every 30 minutes notifying you if the script fails (the voice app stops behaving as expected).
Usability Performance Testing
The main goal of this kind of testing is to identify issues with the speech recognition and NLU behavior of the assistant and your app. It consists of comprehensive testing of the interaction model, creating a baseline set of results. These results, in turn, are the basis for making improvements to the interaction model and the code of the skill — once revisions are completed, additional tests are run to ensure everything is working as expected.
How to get started?
- Provide information about your voice app, the interaction model and the terms to test in this form.
- Wait for the results (you will get an initial grade which will be used as a baseline for next test executions).
- Make improvements to your interaction model and code based on the results got in the previous step.
- Repeat the testing to see how your voice app improves speech recognition before releasing it to production.
Pulling It Altogether
As you can see, testing voice apps is easy and takes little time if you have the appropriate tools. Do not wait any longer. Detect errors in early stages to save money, and launch your voice app with confidence doing end-to-end and usability testing! Remember all these kind of testing is automated, silent, and can be repeated as many times as you need.
Automated testing can help ensure that you are delighting your users — with an approach that is comprehensive and repeatable. To learn more, just contact us and we’ll be happy to setup a demo — or try it yourself. You can sign up for a free trial here.
Yours In Testing,
(*): We have analyzed the most popular skills on Amazon Alexa store.
Don’t forget to give us your 👏 !
Automated Quality Assurance for Voice Apps: Easy, Fast and Affordable. This is How. was originally published in Chatbots Life on Medium, where people are continuing the conversation by highlighting and responding to this story.