Publications & Research

Unpredictable movement patterns and small visual marks make precise tracking of fast-moving tiny objects like a racquetball one of the challenging problems in computer vision. This challenge is particularly relevant for sport robotics applications, where lightweight and accurate tracking systems can improve robot perception and planning capabilities. While Kalman filter-based tracking methods have shown success in general object tracking scenarios, their performance degrades substantially when dealing with rapidly moving objects that exhibit irregular bouncing behavior. In this study, we evaluate various Kalman filter-based tracking methods for fast-moving tiny objects.

While computer vision has advanced considerably for general object detection and tracking, the specific problem of fast-moving tiny objects remains underexplored. This paper addresses the significant challenge of detecting and tracking rapidly moving small objects using an RGB-D camera. Our novel system combines deep learning-based detection with physics-based tracking to overcome the limitations of existing approaches. Our contributions include: (1) a comprehensive system design for object detection and tracking of fast-moving small objects in 3D space, (2) an innovative physics-based tracking algorithm that integrates kinematics motion equations to handle outliers and missed detections, and (3) an outlier detection and correction module that significantly improves tracking performance in challenging scenarios such as occlusions and rapid direction changes. We evaluated our proposed system on a custom racquetball dataset. Our evaluation shows our system surpassing kalman filter based trackers with up to 70\% less Average Displacement Error. Our system has significant applications for improving robot perception on autonomous platforms and demonstrates the effectiveness of combining physics-based models with deep learning approaches for real-time 3D detection and tracking of challenging small objects.

This thesis explores the use of stereo camera for detection and tracking of fast-moving small objects in 3D space. Several Object Detection and Tracking algorithms were experimented. For Object Detection modern YOLO algorithms like YOLOv5, YOLOv7, YOLOv8 as well other multi and single stage detectors were used. Kalman Filter based algorithms were implemented for the tracking purpose. After several experiments with the state-of-the-art models, the results were unsatisfactory, leading to the development of a novel tracking system that augments kinematics to enhance the tracking and trajectory forecasting of fast-moving tiny objects.

Conducted experimental analysis on the learning capability of various deep learning based models like CNN, Compact Convolutional Transformers and Vision Transformers. The goal of the research was to understand the efficiency of larger deep neural networks for image classification tasks on various datasets like CIFAR-10, CIFAR-100, MNIST, Fashion-MNIST, and few other small datasets. The result showed that heavier deep neural networks like ViT, CCT are inefficient in terms of accuracy and speed, while few layers deep CNN model can give significant accuracy as well as speed.

Implementation of Transformer architecture primarily for NLP tasks, focused on Sentiment Analysis and Language Translation. We performed language translation for English-to-French, English-to-Spanish, Portuguese-to-English. Our general transformer achieved decent performance in language translation with acceptable BLEU score, accuracy, and F1 measure for all three language translations. For sentiment analysis, we focused on question answering (Q-A), text classification, and Named Entity Recognition (NER). We achieved exact match score of 78% for Q-A and consistent training accuracy of 40% for the NER task. We also implemented ViT for image classification on CIFAR-10 with test accuracy of 70% and top-5 accuracy of 97%.

We formulate the problem as a sequential decision-making process where an agent is equipped with a conventional object detector to detect objects and the agent is trained to make the detection result better. In this paper, we experiment with using the dynamic method for object detection. Our detector, which uses VGG16 pre-trained on ImageNet as a base, is trained using the PASCAL VOC2012 dataset. The agent takes more actions in transforming the bounding box and the reward accumulated. The AP and recall are best for an IoU threshold of 0.5 or lower. We achieved AP and recall of 36.68 and 59.0 respectively for the 'cat' class at IoU of 0.5.