One of our group members, Haochen, has been working with an optimization-based motion planning framework named Relaxed IK (Inverse Kinematics) in our department's robotics lab. The camera in-hand robot driven by Relaxed IK is able to adapt its viewpoint in visually complex environments. It improves the ability of remote users to teleoperate a manipulation robot arm by continuously providing the users with an effective viewpoint using a second camera in-hand robot arm.
With our vision plugin, a camera in-hand robot will be able to work on a wider range of tasks currently incapable of. For example, the robot manipulator will be able to “look” at the environment with a camera in hand to potentially achieve some objective, such as taking a perfectly stable panoramic picture, seeing around obstacles, and tracking a moving object. To achieve smooth and feasible motions that avoid joint-space discontinuities, self-collision, and kinematic singularities, Relaxed IK casts the standard inverse kinematics formulation into a multi-objective nonlinear optimization problem. A task that requires visual information from the environment can be encoded as one of the aforementioned objectives in this optimization problem. Relaxed IK will be able to effectively make trade-offs and simultaneously reconcile this visual information objective and many other potentially competing objectives.
There exist many methods that control the robot arm to track and grasp a moving object in the environment precisely. However, most of the existing methods don't have the ability to reconcile many potentially competing objectives in real-time. For example, in the case of tracking a moving object, if the robot manipulator is working in a clustered environment or this object is in a trajectory that is going to collide with the robot, these methods will probably fail to avoid the collision, while Relaxed IK will be able to effectively detect the collision and plan around the object with its obstacle-avoiding objective. If successfully implemented, our vision plugin will be a new approach in the sense of reconciling visual information in the environment as an objective in the optimization problem with other objectives such as end effector position/orientation goal matching, minimum-jerk joint motion, and distance from collision states.
We will evaluate the performance both empirically and analytically. For the empirical part, we will design a set of interesting scenarios for testing. For example, we will have the camera in-hand robot track a moving object in a clustered environment to show that it is able to deal with multiple objectives (avoiding collision and tracking the moving object) at the same time. We will also ask the camera in-hand robot to take a panoramic picture of the environment around and see how good the picture is. For the analytical part, we will collect statistics such as average joint velocity, average joint acceleration, and average joint jerk from the joint trajectory returned by relaxed IK and examine how smooth the planned motion is. Another direction to look into is possibly to compare our approach with other available methods of tracking a moving object for the robot arm and analyze its relative robustness and effectiveness.
Get familiar with available packages of doing 3D vision, optical flow and motion measurement.
Implement a method to detect and identify static and dynamic objects in the environment.
Integrate this method into Relaxed IK and make it compatible with other objectives of Relaxed IK.
Design some interesting scenarios to test the vision plugin both empirically and analytically
