A VOICE BASED E-MAIL SYSTEM FOR VISUALLY IMPAIRED PEOPLES
14IT450 ENGINEERING DESIGN
Department of Information Technology
THIAGARAJAR COLLEGE OF ENGINEERING
(A Govt. aided autonomous institution affiliated to Anna University, Chennai)
MADURAI – 625 015
THIAGARAJAR COLLEGE OF ENGINEERING, MADURAI-15
(A Government aided ISO 9001:2008 certified autonomous institution affiliated to Anna University)
Name: K.AKSSHAY KUMAR (16IT006)
Name: M.R.LOGESHWARAN (16IT048)
Name: M.SANJAYBABU (16IT090)
In today’s world communication has become so easy due to integration of communication technologies with internet. However the visually challenged people find it very difficult to utilize this technology because of the fact that using them requires visual perception. Even though many new advancements have been implemented to help them use the computers efficiently no naïve user who is visually challenged can use this technology as efficiently as a normal user can do that is unlike normal users they require some practice for using the available technologies. This paper aims at developing an email system that will help even a naïve visually impaired person to use the services for communication without previous training. The system will not let the user make use of keyboard instead will work only on mouse operation and speech conversion to text. Also this system can be used by any normal person also for example the one who is not able to read. The system is completely based on interactive voice response which will make it user friendly and efficient to use.
TABLE OF CONTENTS
CHAPTER 1 INTRODUCTION
Problem Importance 1
CHAPTER 2 LITERATURE REVIEW
2.1 Literature survey
2.2 Problem impact 3
CHAPTER 3 PROBLEM DESCRIPTION
3.1 Problem Environment
3.2 Problem Definition
3.3 Problem objectives
3.4 Problem Assumptions 4
CHAPTER 4 SOLUTION METHODOLOGY
4.1 Existing System
4.2 Proposed system
4.3 Alternate and finalized ideas
4.4 High level design 6
CHAPTER 5 CONCLUSION 14
Internet is considered as a major storehouse of information in today’s world. No single work can be done without the help of it. It has even become one of the de fact methods used in communication. And out of all methods available email is one of the most common forms of communication especially in the business world. However not all people can use the internet. This is because in order to access the internet you would need to know what is written on the screen. If that is not visible it is of no use. This makes internet a completely useless technology for the visually impaired and illiterate people. Even the systems that are available currently like the screen readers TTS and ASR do not provide full efficiency to the blind people so as to use the internet. As nearly 285 million people worldwide are estimated visually impaired it become necessary to make internet facilities for communication usable for them also.
Therefore we have come up with this project in which we will be developing a voice based email system which will aid the visually impaired people who are naive to computer systems to use email facilities in a hassle free manner. The users of this system would not need to have any basic information regarding keyboard shortcuts or where the keys are located. All functions are based on simple mouse click operations making it very easy for any type of user to use this system.
Also the user need not worry about remembering which mouse click operation he/she needs to perform in order to avail a given service as the system itself will be prompting them as to which click will provide them with what operations.
1.2 PROBLEM IMPORTANCE
A survey shows that there are more than 250 million visually challenge people around the globe. That is, around 250 million people are unaware of how to use Internet or E-mail. The only way by which a visually impaired person can send an E-mail is, they have to dictate the entire content of the mail to a third person ( not visually challenged ) and then the third person will compose the mail and send on the behalf of the visually impaired person. But this is not a correct way to deal with this problem. It is very less likely that every time a visually challenged person can find someone for help. Although for these reasons the specially able people are criticized by our society. So, for the betterment of society and giving an equal status to such specially able people we have come up with this project idea which provides the user with ability to send mails using voice commands without the need of keyboard or any other visual things.
International Journal of Innovative Research in Computer and Communication Engineering.
In this paper we come up with the system which will help to visually impaired people to access email service in hassle free manner which is most universal form of communication in today’s world.
Source with Date (Journal, Magazine, Website)
2.1.2 IJSRD – International Journal for Scientific Research ; Development
Voice based email system helps visually challenged people to access email services efficiently. It has been observed that nearly about 60% total blind population across the world is present in India. This system overcomes difficulties faced by visually impaired people as well as illiterate people. This will reduce the drawbacks of existing system such as software load of using screen readers and Automatic Speech Recognizer (ASR).
Source with Date (Journal, Magazine, Website)
This project is proposed for the betterment of society. This project aims to help the visually impaired people to be a part of growing digital India by using internet and also aims to make life of such people quite easy. Also, the success of this project will also encourage developers to build something more useful for visually impaired or illiterate people. And for also the people who also deserves an equal standard in society.
3.1 PROBLEM ENVIRONMENT
For people who can see, e-mailing is not a big deal, but for people who are not blessed with gift of vision it postures a key concern because of its intersection with many vocational responsibilities. This voice based email system has great application as it is used by blind people as they can understand where they are. E.g. whenever cursor moves to any icon on the website say Register it will sound like “Register Button”. There are many screen readers available. But people had to remember mouse clicks. Rather, this project will reduce this problem as mouse pointer would read out where he/she lies. This system focuses more on user friendliness of all types of persons including regular persons, visually compromised people as well as illiterate.
3.2 PROBLEM DEFINITION
As the title suggests, the application will be a web-based application for visually impaired persons using IVR- Interactive voice response, thus enabling everyone to control their mail accounts using their voice only and to be able to read, send, and perform all the other useful tasks. The system will prompt the user with voice commands to perform certain action and the user will respond to the same. The main benefit of this system is that the use of keyboard is completely eliminated the user will have to respond through voice and mouse click only. But this system will perform actions based on the clicks only that is left click or right click, it does not depends on the portion of the screen where the cursor is placed before the click giving user the freedom to click blindly anywhere on the screen.
3.3 PROBLEM OBJECTIVES
This application proposes an android application, designed specifically for visually impaired people .This application provides a voice based mailing service where they could read and send mail on their own, without any guidance.
Here, the users have to use certain keywords which will perform certain actions such as Read, Send, Compose Mail, Address Book etc. The VMAIL system can be used by a blind person to access mails easily and efficiently. Thus reliance of visually impaired on other people for their activities related to mail can be reduced.
3.4 PROBLEM ASSUMPTIONS
The software developed will be primarily for visually impaired individuals who have no other disabilities. The user will have the physical ability to carry the palm-sized device and walk to their desired end location (destination). The platform off which the software is built will leverage the .NET Framework 3 in order to use voice recognition and text-to-speech functions. Hence, the Operating System used must be Windows. The device automatically connects to wireless networks and does not need any configuration by the user. The end-user is patient with technology and is willing to learn and update himself / herself as technological advances are made.
4.1 EXISTING SYSTEMS
There are a total number of 4.1 billion email accounts created until 2014 and an there will be estimated 5.2 billion accounts by end of 2018.This makes emails the most used form of communication. The most common mail services that we use in our day to day life cannot be used by visually challenged people. This is because they do not provide any facility so that the person in front can hear out the content of the screen. As they cannot visualize what is already present on screen they cannot make out where to click in order to perform the required operations. For a visually challenged person using a computer for the first time is not that convenient as it is for a normal user even though it is user friendly. Although there are many screen readers available then also these people face some minor difficulties. Screen readers read out whatever content is there on the screen and to perform those actions the person will have to use keyboard shortcuts as mouse location cannot be traced by the screen readers. This means two things; one that the user cannot make use of mouse pointer as it is completely inconvenient if the pointer location cannot be traced and second that user should be well versed with the keyboard as to where each and every key is located. A user is new to computer can therefore not use this service as they are not aware of the key locations. Another drawback that sets in is that screen readers read out the content in sequential manner and therefore user can make out the contents of the screen only if they are in basic HTML format. Thus the new advanced web pages which do not follow this paradigm in order to make the website more user-friendly only create extra hassles for these people. All these are some drawbacks of the current system which we will overcome in the system we are developing.
4.2 PROPOSED SYSTEMS
The proposed system is based on a completely novel idea and is nowhere like the existing mail systems. The most important aspect that has been kept in mind while developing the proposed system is accessibility. A web system is said to be perfectly accessible only if it can be used efficiently by all types of people whether able or
disable. The current systems do not provide this accessibility. Thus the system we are developing is completely different from the current system. Unlike current system which emphasizes more on user friendliness of normal users, our system focuses more on user friendliness of all types of people including normal people visually impaired people as well as illiterate people. The complete system is based on IVR- interactive voice response. When using this system the computer will be prompting the user to perform specific operations to avail respective services and if the user needs to access the respective services then he/she needs to perform that operation. One of the major advantages of this system is that user won’t require to use the keyboard. All operations will be based on mouse click events. Now the question that arises is that how will the blind users find location of the mouse pointer. As particular location cannot be tracked by the blind user the system has given the user a free will to click blandly anywhere on the screen. Which type of click will perform which function will be specified by the IVR. Thus user need not worry about location of the mouse at all. This system will be perfectly accessible to all types of users as it is just based on simple mouse clicks and speech inputs and there is no need to remember keyboard shortcuts. Also because of IVR facility those who cannot read need not worry as they can listen to the prompting
done by the system and perform respective actions.
Fig 4.2 E-R Diagram of our system
220.127.116.11 User Interface Design:
The user interface is designed using Adobe Dreamweaver CS3. The complete website focuses more on efficiency in understanding the IVR rather than the look and feel of the system as the system is primarily developed for the blind people to whom the look and feel won’t be of that primary importance as the efficiency of understanding the prompting would be.
18.104.22.168 Database Design:
Our system maintains a database for user validation and storing mails of the user. There are a total of five tables. The relationship between them is assigned after much consideration. The E-R diagram of our complete system is depicted. The Inbox, Sent-Mail and Trash schemas will store all mails of the respective service that belongs to that particular user.
22.214.171.124 System Design:
Fig. 2 depicts the complete system design. It is the level-2 data flow diagram which gives complete detailed flow of events in the system. As we can see all operations are performed by mouse click events only. Also at some places voice input is required.
4.3 ALTERNATE AND FINALIZED IDEAS
This system is currently being developed by us. The following are modules are the ones that are already developed. Their working is as follows:
This is the first module of the system. Any user who wishes to use the system should first register to obtain username and password. This module will collect complete information of the user by prompting the user as to what details needs to be entered. The user will need to speak up the details to which the system will again confirm by prompting alphabetically. If the information is not correct user can re-enter else the prompt will specify the operation to be performed to confirm. Fig. 3. Registration Page ,Welcome Page .
Once the registration is done the user can login to the system. This module will ask the user to provide the username and password. This will be accepted in speech. Speech conversion will be done to text and user will be told to validate whether the details are entered correctly or not. Once the entry is done correctly database will be checked for entry. If the user is authorized it will be directed to homepage.
4.3.3 Forgot Password:
In case where an authorized user forgets the password and thus is not able to login he/she can select forgot password module. In this module the user will be first told to enter username. According to username the security question will be searched in database. This is the question provided at time of registration. The question will be spoken out by the computer. The user should in turn specify the answer that was provided by him/her during registration. If both get matched, user is given option to change password. Forgot Password- accepting user name. Forgot password- Asking security question. Forgot Password- Asking new password.
4.3.4 Home Page:
The user is redirected to this page once log in done successfully. From this page now the user can perform operations that the user wishes to perform. The options available are: 1. Inbox 2. Compose 3. Sent mail 4. Trash Prompting will provide the mouse click operation that needs to be performed for the required service. The double right click event is specifically reserved to log out of the system at any time the user wants to. This will be specified by the prompt right at the beginning after login. Home Page All these functionalities has been implemented.
4.3.5 Compose mail:
This is one of the most important options provided by the mail services. The functionality of compose mail option would not match the already existing mail system. Since the system is for visually challenged people and keyboard operations are completely avoided composing mail would only be done on voice input and mouse operations. No typed input will be required. User can directly record message that needs to be propagated and can send it. This voice massage will go in form of attachment. The receiver can hear the recording and get the message user wanted to send. User would not require attaching the file. Record option will be provided in the compose window itself. Once recorded it will confirm whether the recording is perfect or not by letting the user hear it and if the user confirms it will be automatically attached to the mail.
Fig 4.3.5 Flow chart of Compose-Mail
This option helps the user view all the mails that has been received to his/her account. The user can listen to mails he/she wants to by performing the click operation specified by the prompt. In order to navigate through different mails prompt will specify which operations to perform. Each time the mail is selected the user will be prompted as whom the sender is and what is the subject of that particular mail. Accordingly user can decide whether the mail needs to be read or not or it should be deleted. Deleted mails will be saved in trash section.
Fig 4.3.6 Flow chart of Inbox
4.3.7 Sent mail:
This option will keep a track of all the mails sent by the user. If the user wants to access these mails, this option will provide them with their needs. In order to access the sent mails user will need to perform the actions provided by the prompt to navigate between mails. When the control lands on particular mail user will be prompted as who the receiver was and what is the subject of the mail. This will help the user in efficiently understanding and extracting the required mail.
Fig 4.3.7 Flow chart of sent Mail
This option will keep a track of all the mails deleted by the user. Deleted mails could be the ones from inbox or sent mail. If at any time the user needs to retrieve a mail which was deleted it can be done from this option.
4.4 HIGH LEVEL DESIGN
This project is designed using some set of APIs. SNMTP(simple network mail transmission protocol)has been used for mailing servicing. Voice typing and dictation speech interaction models are designed using the windows 7 LVCSR dictation engine. In order to control speech accuracy we turned off the (default) MLLR acoustic adaptation. Error correction methods are implemented using the windows 7 APIs and windows presentation foundation(WPF).
4.5 PROJECT CONSTRAINTS
It have to be small enough to fit in palm-sized device.
It have to be efficient enough on power to last a whole day, perhaps more as safety precaution.
The device should have a minimum memory requirement of 512 MB RAM.
In this paper we have proposed a system which will help the visually impaired people to access email services efficiently. This system will help in overcoming some drawbacks that were earlier faced by the blind people in accessing emails. We have eliminated the concept of using keyboard shortcuts along with screen readers which will help reducing the cognitive load of remembering keyboard shortcuts. Also any user who does not know the location of keys on the keyboard need not worry as keyboard usage is eliminated. The user only needs to follow the instructions given by the IVR and use mouse clicks accordingly to get the respective services offered. Other than this the user might need to feed in information through voice inputs when specified.
Jagtap Nilesh, Pawan Alai, Chavhan Swapnil and Bendre M.R.. “Voice Based System in Desktop and Mobile Devices for Blind People”. In International Journal of Emerging Technology.
Ummuhanysifa U.,Nizar Banu P K , “Voice Based Search Engine and Web page Reader”. In Internationa Journal of Computational Engineering Research (IJCER). Pages 1-5.
G. Shoba, G. Anusha, V. Jeevitha, R. Shanmathi. “AN Interactive Email for Visually Impaired”. In International Journal of Advanced Research in Computer and Communication Engineering (IJARCCE), 2014 on Pages 5089-5092.
The Radicati website. Online. Available: http://www.radicati.com/wp/wp-content/uploads/2014/01/Email- Statistics-Report-2014-2018-Executive-Summary.pdf.
The WHO website. Online. Available: http://www.who.int/mediacentre/factsheets/fs282/en/