AI Software Framework for 3D Robotic Vision

Name: Gurjeet Singh
Affiliation:
Phone: 5419089519
E-mail: singhg@oregonstate.edu
Website:
Knowledge Required: Students should have basic knowledge of Object Oriented Programming (Java will be helpful). Students should also have knowledge of Python. We can provide help to the students in Deep-learning.
Motivation: 3D robotic vision is used in many next-generation technologies: self-driving vehicles, AR/VR, robotic grasping and manipulation, and autonomous driving. Unfortunately, current computer vision techniques using deep-learning are only trained on 2D-image databases. There is a need to build-up a new 3D training database, especially for long-distance point clouds (i.e. using LIDAR). Finally, new deep-learning algorithms based upon this 3D-mapping will need to be developed.
Description: A 3D-mapping database, software-interface, and deep-learning platform are desired for this computer vision project. Students will utilize currently-available 3D-cameras to generate a large training database, and develop new 3D-deep learning algorithms, based on previously completed research from Professor Chiang's research team members.
Objectives: Create a software framework (utilizes pre-existing code-base) that captures data from previously-existing HW camera and LIDAR, generate new 3D-image database, and perform semantic segmentation, training, and object detection and classification.
Deliverables: Team must provide framework code ( Google Project-Tango is preferred, as we have already written some preliminary code-base). Depth sensing hardware (i.e. LIDAR) will be provided by Professor Chiang's Research Group. The software framework should perform the following operations:
1. Collect data from already-existing hardware (LIDAR, HD RGB-Cameras and other off-the-shelf depth sensors)
2. Collect large 3D data-set from existing hardware setup, generate local training database, and create ground-truth (labeling; segmentation) from data.
3. Use open-source Deep-learning platforms (i.e. Microsoft Faster R-CNN or Facebook Mask R-CNN) to perform object detection and semantic segmentation. Our research group at OSU already has preliminary default pre-trained neural networks that you can use, as a starting point and reference.
Other comments:

   D. Kevin McGrath
   Last modified: Thu Nov 16 11:32:03 2017