Build basic VR Unity web application using HTC Vive headset and hand trackers to control a virtual robot arm
-Build 3-4 different environments (warehouse picker station, bathroom, conveyor belt with items flowing by, clothes on table for folding)
Each environment should have the ability to randomize background parameters (colors, shape, size, weight of objects and surroundings) -> See domain randomization link
-Integrate imported CAD models for ABB YuMi, or Qoowala robots
-Interface with ROS (or perform Unity calculations) to solve robot kinematics in real-time corresponding to hand trackers
-Build a control/command GUI interface in VR with different control modes
Homonculus view with camera feeds
Various hand-controller button options for gripper control and orientation for comfortable/intuitive teleoperation
-Build framework for recording/exporting real-time data during simulation
Export data such as:
Timestamp, VR rendered image as .jpg, 6-dof object poses in environment, robot joint angles, robot end-effector pose, gripper state
This is project description and I'd like to get started within tomorrow.
Please break down your milestones in detail with your proposal.