Vision based pick and place Robotic arm

This blog explains the basic working of a robotic arm under vision system using matlab
For this you need to know basic programming in matlab and arduino.

Items Used:
Dexter ER-2 Heavy duty robotic arm 

Webcam 

Arduino Uno

custom made arduino shield used to mount servos

Plastic ball (plain in color)

Flex sheet with polar and rectangular coordinate systems

SMPS for the robotic arm

USB Cable

Dexter ER-2 Heavy Duty Robotic Arm is a 5 Axis robotic Arm + Servo Gripper. It uses seven metal gear servo motors with 15Kg/cm torque and two servo motors with 7Kg/cm torque. Robot Arm has 5 degrees of freedom which includes Base rotation, Shoulder rotation, Elbow rotation, Wrist pitch and roll. Out of which Shoulder rotation, Elbow rotation, Wrist pitch has two 15Kg/cm torque servo motors in parallel for enhancing the performance of the robotic arm.

The servos of the arm are controlled using arduino pwm(pulse width modulation) signals and the main power supply is commonly distributed  for all the servos using a shield which is powered by SMPS. Here we are using arduino for easy interface with matlab and plain colored plastic ball for easy processing of the image, note to choose ball color distinct from its background. You can find RGB values of the ball by image analysis in matlab and can apply basic thresholding technique to threshold the ball from the background and can find one in my code and lighting in the room place a major role when you are taking RGB reading and make sure light does not change much in the room. Then apply a high pass filter mask on the image to find edged image from which finding circles is easy using an inbuilt command imfindcircles in matlab. The concept behind the command can be found in matlab blogs. This command returns the center location of the found circle.

Now comes the inverse kinematics in action, we can find desired angles for the arm to reach the ball by placing center pixel value of the ball in inverse kinematics equations. Matlab serially transmits the angles to arduino using serial communication and arduino does the rest of the work in giving required pwm signals to arm servos to pick the ball.

I used Giampiero Campa's code to interface Arduino and matlab, you can get the code here

Shield used for connecting all the servos of the arm to arduino 

For more information on inverse kinematics using matlab visit http://www-iri.upc.es/people/thomas/papers/PROBLEMS.pdf 

Matlab codes:
https://github.com/Akhilkumar1307/Vision-based-pick-and-place-robotic-arm    

Working video

comment below for any enquiries