PONTA, the robot that understands you!

@rupam-das-wW5CvN • Oct 22, 2024

Oct 22, 2024

1.1K

[NOTE: This page is strictly copyrighted to crazyengineers.com and grasshoppernetwork.com. Copying, reusing any content from this page is strictly prohibited. Any violation will draw attention from DMCA. Be advised!]

We at Integrated Ideas Consultancy Services, have announced project PONTA officially on 15-05-2013. PONTA is a robot which is certainly first of it's kind.

This is a research project that will have following features
It is a robot that listens to your, that can hear you, that can understand you and does stuff based on what you tell 'HIM' to do or direct him to do. Just like perceptual computing, Robotics is going to change our lives in very near future. Replace virtual reality game with physical robots where robot can play football, it can bring you water.
So what PONTA is:
1. It is a Lighweight Robotic suite with main components as: Robotic Arm, Controller Circuit, Arduino for Controlling and Perceptual computing Interface for transferring Human action to Robot understandable actions.

2. What it does: It is equipped with two arms with 5DOF for each arm. It will be able to dance, fight, lift things, and move. It will be mounted on a rover and Perceptual camera will be mounted on top of rover. So it can move around, look around and listen around.

3. Senses: Huh? PONTA will sense temperature, humidity, light intensity, moisture, smell and many other physical entities. And of course it will sense you. Your voice, your actions.

So where will it see itself being used?

1. Personal Entertainment: You would really love to see a robot dancing with music rather than an animated character on your screen. Right?

2. Surveillance: We will configure It with surveillance capability using face recognition. So it can move around, can capture faces around, can track if any face is in naughty database and trigger alarm.

3. Robotics Gaming: There are increasing number of Robotic Olympics now. Forget controlling your robot with Silly joystick. Ask it to come close to you and it will.

4. Personal Assistance: Ya? Yes. So our beloved PONTA will be faciliated with watering the plants. You ask it to do, it would do. It will move closer to tubs, check moisture and sprinkle suitably.

It will be a robot that will understand human action that no robot had ever done in history.

This will be almost certainly an open source project and technical details will be released in grasshoppernetwork.com and this page of crazyengineers.com. Also find us in facebook:
#-Link-Snipped-#

Stay tuned to this thread for regular updates about the project. Wish us luck. Let the madness begin.

Updates:-

#-Link-Snipped-#

Like 0 Replies 17

Replies

Welcome, guest

Join CrazyEngineers to reply, ask questions, and participate in conversations.

CrazyEngineers powered by Jatra Community Platform

Anoop Mathew

@anoop-FRTf1L • May 14, 2013

All the best Rupam Sir! Why the name PONTA?
0
Rupam Das

@rupam-das-wW5CvN • May 17, 2013

Because my son called the prototype "Ponta"
0
Rupam Das

@rupam-das-wW5CvN • May 17, 2013

Day 3, Ponta Update: Testing voice commands:

It uses Microsoft's new speech SDK for detecting and recognizing voice.
0
Rupam Das

@rupam-das-wW5CvN • May 25, 2013

Ponta, Day 10, Week 2

Ponta is now responding to voice, head movement and hand gestures at the same time.

It's day ten since we started with Ponta project. Things are as rough as we thought they would be. We have used OWI kit so far. It is closest to one used in practical robot and industries. I wanted to avoid Lego at all costs. Arduino gives me better flexibility than Lego. One more problem I faced with it is integrating two cameras. I want to fit a camera in the robot and other might be used for remote control. So I did a little work to get two camera's working simultaneously with the interface.

So at second week Ponta has started responding to voice, face ( head ) movement and hand gestures. The response so far is not really outstanding and lot of polishing and filtering is required at this stage to get a clean nice response from the robot. However seeing it responding to all modalities is still a great fun.

There is one thing though! Ponta is not yet detecting and recognizing objects. I want it to recognize spoons, tea pot and so on. I want it to identify and get the things of it's own rater than waiting for me to give command.

So I would work with object detection. I am not sure if the SDk provides object recognition! If it does not, then I will use simple k-nearest neighbor classifier rather than using something like SVM. KNN will save time. If detection is perfect, recognition shouldn't be any problem.

I would appreciate your critics and suggestions.
0
Anoop Kumar

@anoop-kumar-GDGRCn • May 25, 2013

Ponta does awesome job..
I have one suggestion that
while moving its body part, it should continue to move until given command OK.
For gesture sensing, It may be possible that it will continue until it see a still open palm.😒
0
Rupam Das

@rupam-das-wW5CvN • May 25, 2013

Thats what it is going to do ultimately. As the sensing is not at all anything closer to accurate, I am sticking to restricted motion to save the poor robot from being gone!
0
Rupam Das

@rupam-das-wW5CvN • May 25, 2013

A night's coding was all needed to integrate basic object tacking. PerC does not come with an object tracking. So I have mapped the depth map image and RGB image, find out the distance between both sensors and aligned feed from rgb and depth sensors. Now, PerC does not give accurate World image coordinates. But the depth values changes with change in Z. Hence I put a simple technique of analyzing the pixel value of depth data and then filter values bellow 240, which return objects closer to camera. Now based on the bounding box of this depth data, I segment the RGB image. This segmented image is acting like the training image.

Next I use GPU based SURF detector to find the location of the object in the scene and threshold on returned Polyline area of remove small area which are result of misdetection. I can say, the overall detection is of about 60% accuracy.

I would now make some changes in the algorithm. Rather than checking each frame, I will first track 20 frames and the maximum area returned will be taken as basis point. From next frames it should search around basis point rather than whole image.
0
Anoop Mathew

@anoop-FRTf1L • May 26, 2013

Awesome videos Sir! I wonder if you'd keep a backup plan in case your robot starts to back-tag you like in the case of Chitti or iRobot. Anyways, you're quite an inspiration to the rest of us. Hats off!
0
Rupam Das

@rupam-das-wW5CvN • May 27, 2013

Day 12, Ponta Project! It's Dancing Now

Though movement is little rusty and mechanical at this moment, we are being able to feed complex sequencial movement to robot. It works with beat classification. You can still enjoy it's steps.
0
Harshad Italiya

@harshad-ukH5ww • May 27, 2013

#-Link-Snipped-# Great going !

It would be great if you share some information about hardware that is being used in PONTA.

0
Anoop Mathew

@anoop-FRTf1L • May 27, 2013

Harshad Italiya
#-Link-Snipped-# Great going !

It would be great if you share some information about hardware that is being used in PONTA.
#-Link-Snipped-# sir, please don't share useful info about the robot publically; don't want the Japs to peek into Indian innovation. 😉

0
Harshad Italiya

@harshad-ukH5ww • May 27, 2013

anoopthefriend
#-Link-Snipped-# sir, please don't share useful info about the robot publically; don't want the Japs to peek into Indian innovation. 😉
I am not saying to share schematics and everything. I know how much hardwork #-Link-Snipped-# is doing. I just want to know what are the core parts in this PONTA. Just curious to know about it's processing power.

0
Rupam Das

@rupam-das-wW5CvN • May 27, 2013

From the word go, Ponta is Open Source. I have been writing about every method I am working on. Here is an update on hardware.
0
Rupam Das

@rupam-das-wW5CvN • May 27, 2013

Ponta is not run by any purchased boards or Motor shields. It is having 5DOF + 1 Base motor. All are geared motors. So all we needed was to control the motors. And by the way, it is envisioned for scalability. So We want to abstract operating voltage. The design that works for 12v motors should also work for 24v motors.
0
Abhishek Rawal

@abhishek-fg9tRh • May 28, 2013

#-Link-Snipped-# Why don't you upload source-code in Github ?
And once completed, the entire project with schematics in Instructables & Ignite Intent ?

0
Rupam Das

@rupam-das-wW5CvN • May 28, 2013

I will keep updating about challenges, methodology and techniques used. However I don't think Github or Instructable will ever come into play. Any part that I plan to release will be released in grasshoppernetwork.com .

However it won't be before September for sure. We will wait till stable release of both hardware and software.
0
Rupam Das

@rupam-das-wW5CvN • May 29, 2013

Day 14, End of Week 2,Simulation Environment added to Ponta

Honestly, the interface and functionality was looking dull and incomplete without a simulation. Not everybody might be having a physical hardware. People may want to test the response virtually first and then opt for a hardware. On the contrary, the are experiments like training dance steps which may damage the gears of actual robot. So I came up with simple 5DOF arm simulation. Simulation will also respond to commands. Commands given to simulation will be replicated in original hardware and vice verse.

So if you want to know how does the arm behave to lift an object from certain coordinates, then nothing better than simulation. Beside it adds amazing capabilities to control as well as interface.

Having get most of the components that we wanted the project to have, we are heading for another complicated phase!

So far we have mapped hand movements to commands and translated the command to robot. It is now precise coordinate system. Arduino would be programmed to move multiple joins together and just like hand.
0