Research Subt-MRS Dataset This project was an year-long endeavor, where I co-led a team of 5 in acquiring a 2 TB SubT-MRS SLAM dataset, collected on 3 UGVs and 1 Boston Dynamics Spot, 5 sensors including RGB camera, thermal camera, IMU, and LIDAR, and 5 locations with different characteristics As a part of data analysis, I implemented and evaluated 4 state-of-the-art SLAM algorithms, LIO-SAM, LOAM, FAST-LIO, and CLINS on the dataset to gauge the difficulty of the dataset I also designed multiple Python3 scripts for dataset cleaning and statistics generation. I also implemented a PyTorch DataLoader for the dataset and integrated it into a pip package I integrated real-time point cloud colorization into the SuperOdometry algorithm by field-of-view analysis between RGB camera and LIDAR for improving visualizations This dataset will be used to host the ICCV 2023 Workshop on Robot Learning and SLAM Open-Set Vocabulary Inter-Object (O2O) 3D Affordance Prediction in Point Clouds Here, we enabled natural language query-driven affordance prediction in point clouds, reaching 90% performance relative to the SOTA of closed-set affordance prediction I also performed ablation study for different point cloud architectures like PointNet, PointTransformer, and LLM architectures like CLIP, CG3D Currently we are working on advancing this object-based affordance prediction to open-set vocabulary-based inter-object prediction, utilizing sim-in-the-loop training Communications and Systems Engineering I developed a C++ algorithm for generating a graph using the Signal-to-Noise Ratio (SNR) values from radios in a multi-robot setup, using Websockets and ROS-DDS Using this graph, I innovated constraints for robot motion planning by utilizing spanning trees on this SNR graph, resulting in a communication network construction algorithm. This increased exploration distances by 90% against SOTA I also devised 2 behavior trees for switching between 8 autonomy modes in the MMPUG multi-robot system with 3 UGVs and 2 Spots Among other systems engineering tasks, I integrated and calibrated a FLIR-Boson thermal camera into the existing sensor stack of 6 RGB cameras, 1 IMU, and a LIDAR with <25 ms time sync delay and 60 Hz frequency Academic Neural Radiance Fields (NeRF) In this project, I implemented 3D volume and surface rendering pipelines, using NeRF The project was implemented in Python3. I also extended the baseline to improve surface reconstruction with realistic light shading using Phong Relighting and improved radiance computation using heirarchical point sampling Learning-based Single View to 3D Here, I designed a Single view to 3D reconstruction pipeline, for image-to-voxel, image-to-mesh, and image-to-point cloud generation I achieved an F1-score of 0.88 for the point cloud reconstruction and also extended the network for occupancy queries Absolute Conic-based Single View to 3D This project deals with single view to 3D reconstruction using classical geometric vision concepts. Here, we first design a single image-based camera intrinsics calibration module, using the concepts of Image of Absolute Conic Using this baseline calibration technique, we design the 3D reconstruction pipeline, using manual plane annotations Multi-View Reconstruction In this project, I built a structure-from-motion (SfM) pipeline from scratch, employing concepts such as perspective-n-point, 8-point algorithm, and traingulation In addition to SfM, I also extended it to include Bundle adjustment, to improve camera extrinsics computation Feature-Extractor Voting for Visual-Inertial Odometry In this project, I incorporated SuperPoint feature extractor into the backend optimization of VINS-MONO We created a feature voting and fusio algorithm to bridge SuperPoint visual features and classical features, whcih led to reduction of odometry failure cases by ~40% MNIST Diffusion Here, I designed a Denoising Diffusion Probabilistic Model (DDPM) for generating the MNIST data The model was written in Pytorch and produced 80% generation for 36 sample sets of the MNIST numbers, using 10000 diffusion steps Point Cloud Classification and Segmentation Here, I designed deep learning-based point cloud classification and segmentation models, including PointNet, PointNet++, DGCNN, and PointTransformer Point Cloud classification was achieved with an average accuracy of ~95% whereas segmentation was achieved with an average accuracy of ~98%. Ablation studies with point cloud transformations were also performed
