1. Home >
  2. Apps >
  3. Projects >

Smart Virtual Assistant Bot Using Voice Control with Raspberry Pi

Voice Activation and Control to Improve Human Robot Interactions with IoT Perspectives YouTube link : https://youtu.be/JVVBGBXeLHU
VIGNESH B.R
Rank D3 - MASTER
₹10000 · Coimbatore · Electronics-And-Communication

Project Abstract

  1. A voice-controlled robot that uses artificial intelligence.
  2. Human voice commands are given to the robotic assistant remotely by using a smart mobile phone or a raspberry pi micro controller or a computer. 
  3. The robot can perform different movements, displaying the temperature and humidity of the environment.
  4. The robot also locates an object when moving from one place to another. 
  5. The speech signal commands converted to text form are communicated to the robot over a Wi-Fi network. 
  6. The camera is connected to Raspberry Pi for detection of objects that is then streamed to the user’s mobile or computer. 
  7. Used for surveillance purpose, monitor the army base, traffic monitoring and human machine interaction.
  8. The smart robot which is made to monitor and check the situation of your home live from remote place through the internet by google assistant and   artificial intelligence   

Proposed System/Methodology 
  • Robot can perform basic movements, detecting temperature and humidity along with detecting objects in real time to interact with the user. 
  • NodeMCU controller-based voice recognition method is displayed in a smart phone. 
  • The trigger service is ‘Google Assistant’ and the Action will be implemented through this service. 
  • The input from the user is through Google Assistant.
  • The raspberry pi module is used for object detection in real time and is controlled by the user’s phone or computer.
Modular Description

The following are the modules in the system :

  •    Voice Recognition Integration 
  •    Robot Movement and Sensor Usage
  •    Object Detection 
  •    Interactive Virtual Assistant
Smart Virtual Assistant Bot Using Voice Control with Raspberry Pi

List of Deliverables

Module 1: Voice Recognition Integration

• Integrating the google assistant as a trigger to the actions on the IFTTT(If This Then That) platform through the Adafruit cloud sever.
• The Adafruit cloud server consists of the data from the device which is displayed to the user.
• Each of the commands through the Google Assistant invokes an action specified in the IFTTT platform.

Module 2: Robot Movement and Sensor Usage

• Movement of the robot based on the commands from the server.
• The temperature and humidity of the area is also displayed in the LCD by the DHT11 sensor.
• Each of the movements are executed by the driver shield that is connected to the NodeMCU unit. 
• The 5V of power supply from the NodeMCU micro controller that is sufficient for the whole circuit.
• The movements are also displayed on the 16x2 LCD display unit each time when a command is issued and when the rover performs the action.


Module 3: Object detection

• Program in python language using open CV library and tensor flow on raspberry pi.  
• Detecting the object given direct via programing by camera. 
• Datasets are preloaded on the file and the extraction is made from the preloaded images.
• Utilization of open source equipment and that chips away at the premise of visual information caught from an ordinary camera which has a reasonable lucidity.  z

Module 4: Interactive Virtual Assistant

• Alexa is also integrated into the raspberry pi module which uses speech to answer queries and respond to tasks.
• A speaker is connected to the module to enable audio streaming.
• Dynamic interactivity enabled.
Platform/Data set/Tools
• IFTTT (If This Then That) control services
• Adafruit cloud server
• NodeMCU
• Raspberry Pi
• Motor shield for NodeMCU
• LCD Display(16x2)
• Camera for raspberry pi(night vision)
• VNC viewer and Mobile SSH(client/server)
• Lithium ion battery
• DH11 sensor(temperature/humidity)
• Speaker


Conclusion

• This robot has multiple features which will give more security to any place and viewed on a mobile phone/computer. 
• There are varieties of sensors for analyzing the current situation, a lot more can also be added according to the need. 
• Everything can be controlled and monitored using mobile or a computer through the internet from any part of the world. 
• The Voice Command System has an enormous scope in the future like Siri, Google Now and Cortana become popular in the mobile industry. 

Tags

#Electronics #raspberry pi #Image Processing #Open CV #NodeMcu #iot #robotics #Mechanical #Adafruit