PONTA, the robot that understands you!

Replies

• 

  Anoop Mathew

  All the best Rupam Sir! Why the name PONTA?
• 

  Rupam Das

  Because my son called the prototype "Ponta"
• 

  Rupam Das

  Day 3, Ponta Update: Testing voice commands:
  
  
  It uses Microsoft's new speech SDK for detecting and recognizing voice.
• 

  Rupam Das

  Ponta, Day 10, Week 2
  
  Ponta is now responding to voice, head movement and hand gestures at the same time.
  
  
  
  It's day ten since we started with Ponta project. Things are as rough as we thought they would be. We have used OWI kit so far. It is closest to one used in practical robot and industries. I wanted to avoid Lego at all costs. Arduino gives me better flexibility than Lego. One more problem I faced with it is integrating two cameras. I want to fit a camera in the robot and other might be used for remote control. So I did a little work to get two camera's working simultaneously with the interface.
  
  
  So at second week Ponta has started responding to voice, face ( head ) movement and hand gestures. The response so far is not really outstanding and lot of polishing and filtering is required at this stage to get a clean nice response from the robot. However seeing it responding to all modalities is still a great fun.
  
  There is one thing though! Ponta is not yet detecting and recognizing objects. I want it to recognize spoons, tea pot and so on. I want it to identify and get the things of it's own rater than waiting for me to give command.
  
  So I would work with object detection. I am not sure if the SDk provides object recognition! If it does not, then I will use simple k-nearest neighbor classifier rather than using something like SVM. KNN will save time. If detection is perfect, recognition shouldn't be any problem.
  
  I would appreciate your critics and suggestions.
• 

  Anoop Kumar

  Ponta does awesome job..
  I have one suggestion that
  while moving its body part, it should continue to move until given command OK.
  For gesture sensing, It may be possible that it will continue until it see a still open palm.😒
• 

  Rupam Das

  Thats what it is going to do ultimately. As the sensing is not at all anything closer to accurate, I am sticking to restricted motion to save the poor robot from being gone!
• 

  Rupam Das

  
  
  A night's coding was all needed to integrate basic object tacking. PerC does not come with an object tracking. So I have mapped the depth map image and RGB image, find out the distance between both sensors and aligned feed from rgb and depth sensors. Now, PerC does not give accurate World image coordinates. But the depth values changes with change in Z. Hence I put a simple technique of analyzing the pixel value of depth data and then filter values bellow 240, which return objects closer to camera. Now based on the bounding box of this depth data, I segment the RGB image. This segmented image is acting like the training image.
  
  Next I use GPU based SURF detector to find the location of the object in the scene and threshold on returned Polyline area of remove small area which are result of misdetection. I can say, the overall detection is of about 60% accuracy.
  
  I would now make some changes in the algorithm. Rather than checking each frame, I will first track 20 frames and the maximum area returned will be taken as basis point. From next frames it should search around basis point rather than whole image.
• 

  Anoop Mathew

  Awesome videos Sir! I wonder if you'd keep a backup plan in case your robot starts to back-tag you like in the case of Chitti or iRobot. Anyways, you're quite an inspiration to the rest of us. Hats off!
• 

  Rupam Das

  Day 12, Ponta Project! It's Dancing Now
  
  
  
  Though movement is little rusty and mechanical at this moment, we are being able to feed complex sequencial movement to robot. It works with beat classification. You can still enjoy it's steps.
• 

  Harshad Italiya

  #-Link-Snipped-# Great going !
  
  It would be great if you share some information about hardware that is being used in PONTA.
• 

  Anoop Mathew

  Harshad Italiya

  #-Link-Snipped-# Great going !
  
  It would be great if you share some information about hardware that is being used in PONTA.

  #-Link-Snipped-# sir, please don't share useful info about the robot publically; don't want the Japs to peek into Indian innovation. 😉
• 

  Harshad Italiya

  anoopthefriend

  #-Link-Snipped-# sir, please don't share useful info about the robot publically; don't want the Japs to peek into Indian innovation. 😉

  I am not saying to share schematics and everything. I know how much hardwork #-Link-Snipped-# is doing. I just want to know what are the core parts in this PONTA. Just curious to know about it's processing power.
• 

  Rupam Das

  From the word go, Ponta is Open Source. I have been writing about every method I am working on. Here is an update on hardware.
• 

  Rupam Das

  Ponta is not run by any purchased boards or Motor shields. It is having 5DOF + 1 Base motor. All are geared motors. So all we needed was to control the motors. And by the way, it is envisioned for scalability. So We want to abstract operating voltage. The design that works for 12v motors should also work for 24v motors.
• 

  Abhishek Rawal

  #-Link-Snipped-# Why don't you upload source-code in Github ?
  And once completed, the entire project with schematics in Instructables & Ignite Intent ?
• 

  Rupam Das

  I will keep updating about challenges, methodology and techniques used. However I don't think Github or Instructable will ever come into play. Any part that I plan to release will be released in grasshoppernetwork.com .
  
  However it won't be before September for sure. We will wait till stable release of both hardware and software.
• 

  Rupam Das

  Day 14, End of Week 2,Simulation Environment added to Ponta
  
  
  Honestly, the interface and functionality was looking dull and incomplete without a simulation. Not everybody might be having a physical hardware. People may want to test the response virtually first and then opt for a hardware. On the contrary, the are experiments like training dance steps which may damage the gears of actual robot. So I came up with simple 5DOF arm simulation. Simulation will also respond to commands. Commands given to simulation will be replicated in original hardware and vice verse.
  
  So if you want to know how does the arm behave to lift an object from certain coordinates, then nothing better than simulation. Beside it adds amazing capabilities to control as well as interface.
  
  Having get most of the components that we wanted the project to have, we are heading for another complicated phase!
  
  So far we have mapped hand movements to commands and translated the command to robot. It is now precise coordinate system. Arduino would be programmed to move multiple joins together and just like hand.