Computer Vision (CV) is one of those APIs which is considered as the essence of Cognitive Services. It provides state-of-the-art image processing algorithms help you moderate content automatically and build more personalized apps by returning smart insights about faces, images, and emotions.
I will be writing more about all the facets individually with appropriate and best-selling use-cases in upcoming posts. Meanwhile, you can have a look at my recent post on Chatbots Magazine which covers an overview of most of the Cognitive Services.
Custom Vision is a subset of Computer Vision. It allows you to customize computer vision models for your unique use case by just uploading a few labeled images and training image classifiers.
As per Microsoft’s official documentation:
Custom Vision Service is a tool for building custom image classifiers, and for making them better over time. For example, if you want a tool that could identify images of “Daisies”, “Daffodils”, and “Dahlias”, you could train a classifier to do that. You do so by providing Custom Vision Service with images for each tag you want to recognize.
Microsoft’s Cognitive Service team has done a great job in bringing up most of the documentation. However, as it is not being widely used among developers, therefore you might encounter lack of support on this.
Custom Computer Vision in Bots
Bots can use the power of Computer Vision in order to detect, identify and authorize your customers and the contextual information which is passed to it.
For example, you own a Cake Parlor and you prepare custom cakes of your customer’s choice. You’ve got a bot that requests for the pictures from your customers. In order to speed up your process, you can train your model to recognize the cake images so that it doesn’t book any order for anything which is not cake.
Another example could be related to Immigration Customer Service Desk where a bot is deployed and it issues a token of specific counter on the basis of various nationalities. You train your custom vision model by providing it images of several passports so that it could direct customers to the appropriate counter.
There could be many other examples but keeping the length of post in mind, I would now give my use-case which I will be using in my own bot that I created in my last post and showed how can we build a multipurpose bot.
I will be continuing using that bot using the Broadband scenario with LUIS.
Scenario / Use case
Let’s say you own a broadband services company and you want to provide a facility to your users to apply for broadband packages online to get 10% discount either through Facebook page or Skype.
Both of these channel deploys the same bot which requests customers some basic information such as Name, Address and Phone number.
I’m not going to write the logic for above in this post as it is quite simple and you can try on your own and feel free to contribute. I will only be writing the code to handle attachments in the form of IDs.
After the initial information, bot requests the customer to verify his identity by providing the scanned image or photo of his ID.
If the computer vision classifier recognizes the expected ID (what it was trained for), bot sends the message to customer that he’d be requested by one of the staff later as staff needs to validate the ID number etc.
In my next post, I will tell you guys on how can we completely automate this verification process by reading the verifying the ID number within the our library.
I have kept it quite simple as my purpose was to onboard you on this unique yet powerful service. You can think of many other better solutions using Custom Vision API.
As an active contributor to the bot community, I have published the source to GitHub for you to understand and contribute should you want to add new features to it.
Until next time, happy coding!