My Projects

I helped create the head-worn display (HWD) exhibition as part of the Contextual Computing Group led by Prof. Thad Starner. The exhibition's main goal was to teach attendees some of the biggest challenges of making HWDs for everyday wear. Attendees experienced live demos of the most popular HWDs of recent years and learned what the headsets did right and where challenges still remain. Attendees also received guided tours of one of the world’s largest personal collections of HWDs and learned how the field has transformed over the last 30 years. I along with my team created interactive demos on various headsets such as the Magic Leap, Hololens 2, Epson BT-30 and Google Glass EE2 to showcase the challenges previously mentioned. The exhibition was presented at ISWC 2022 and GVU Expo 2022 among other events. I along with my team have been continuously working on updating the exhibition and generally present it once every few months. To find the date and time for the next showcase, feel free to contact me. To learn more, click on the link below.

As part of my role as a student researcher at the contextual computing group, I have been tasked to conduct various studies that test certain applications of HWDs as well as find challenges in using them for everyday wear. These studies are intended to build towards publications. Two of the most resent studies have been on Remembrance Agent and Safety. Remembrance agents are conversational aids that provide just-in-time information. Subtle head-up displays (SHUDs) can be used as these conversational aides, coaching wearers with conversational topics or just-in-time information cued by speech transcribed by modern recognition systems. SHUDs are also becoming more difficult to distinguish from normal eyeglasses. However, would the use of such smartglasses break the rapport between conversants? Would such aides actually improve conversations? Our studies on remembrance agent aim to address these questions. With safety, our research aims to find the safest position in the visual field for using HWDs while walking. Participants wore a Microsoft Hololens 2 which will be used to simulate the different HWD positions in the visual field. Participants were tasked to read text on the HWD while walking around a path to the beat of a metronome to emulate a scenario of using HWD in daily life. We flashed the screen white and recorded the participant's read speed, natural walking speed, as well any missteps during the experiment.

Augmented Reality for Visually Impaired People uses Augmented Reality devices (the Microsoft Hololens) and a combination of spatial audio clues and speech sounds to deliver information about a user's surroundings. The project delivers spatial audio pings so users can understand where walls and obstacles are, as well as reads out any text in the user's environment. Our team conducted a user study with 7 blind users and found promising results. I worked on AR for VIPs during my semester abroad at UC Berkeley as part of the Extended Reality @ Berkeley group. My main responsibilities were object sonification, which involved automatically breaking down the Mesh generated by the Hololens to distinguish obstacles, and text recognition. The project was presented at the Microsoft Reactor in San Francisco and I later presented it as a poster at India HCI 2019. A demo video and the link to the project site can be found below. You can also check out my poster for India HCI here: https://bit.ly/AR4VIPS

ScholAR is an ongoing project at IMXD Lab, IIT Bombay focused on designing AR learning experiences for middle school students in rural India. ScholAR aims to encourage collaboration in students and improve their visualization and spatial conceptualization. The project consists of two applications, a teacher-side app and a student-side app. The teacher-side app helps the teacher monitor and interact with the entire class while the student-side host the education modules. I worked on ScholAR for 6 months as a research intern at IMXD Lab. I mainly worked on networking and development of the teacher-side application. You can learn more about ScholAR at the link below.

This project aimed to highlight performance differences between two very different GPGPU (General-Purpose computing on Graphics Processing Units) techniques, CUDA and Compute Shaders (Unity). These techniques are used widely in the video game industry for paralleling computational tasks that can easily be divided into smaller tasks. We conducted simple N-particle simulations as out benchmark to depict the performance difference of the two approaches. This white paper was written for UC Berkeley's CS267 Applications of Parallel Computers. The complete white paper can be found on the link below.

We implemented a communication system using BCI targeted for people suffering from partial paralysis or brain damage. It is aimed to be an independent communication system that allows users to control a speller and a drawing scene using a BCI and gyroscopes. We then created a CNN to predict user's drawings and used text-to-speech to describe what the user drew. We also experimented with using Virtual Reality to create a controlled environment for recording the EEG signals. We then analyzed both the recorded signals and checked if the use of virtual reality results in less noisy data corresponding to less amount of spontaneous EEG. This research project was my major/capstone project for my Bachelors degree and culminated in a peer-reviewed publication in an international journal. You can read the complete paper on the link below.