2025 Conference on Robot Learning (CORL 2025, 35.77% Acceptance Rate)

This is my first real research project that I have taken full ownership of. As the only research student under my supervisor, Dr. Daniela Constantinescu, I was given the oppurtunity to define my own research path. After some discussions with Dr. Constantinescu, we decided that my interest in reinforcement learning and her expertise in control theory, the implementation of Lyapunov functions in reinforcement learning would be a great starting point. Ultimately, this led to my first publication which has been accepted to CORL 2025!

This paper looks at model-free reinforcment learning problems in robotics and attempts to provide a stability certificate for the final controller. The main contribution of this paper is that we are able to learn a neural Lyapunov function off-policy. This provides a significant improvement when sample efficiency is crucial.

The preprint of the paper can be found here.

After quitting my job, I had some more time to learn and improve my skills in areas I was interested in. One of the things that I always wanted to learn was game development. So I took it upon myself to learn Unity and C# while building a 2D platformer. This is my ongoing project that I have been working on for a couple months now. I took a little bit of a break to do some travelling but the project is now back in full swing and the first room is pretty much complete as shown below!

Some of the things that I have learnt and implemented are simple UI and health, player/enemy interactions, animations, and triggers. However, my largest focus has been on player movement. I currently have basic running, sliding, jumping, double jumping and wall jumping implemented. I spent about 2 weeks fine tuning the wall jumping and after that I had conisdered this first room complete. My goal was to complete this room and then determine a new goal.

Next Step - GBA Development

After completing the first room of my Unity game, I felt like I had a very basic grasp and understanding of game dev and some of the challenges. So I wanted to set myself a more well defined goal. I also want to include my interest in firmware as well. I spent a couple of weeks thinking about how I could do this trying out a variety of ideas like creating physical controllers to an arcade cabinet. What I eventually settled on was creating a game for the Gameboy Advanced.

Why the Gameboy Advanced? The GBA uses an ARM7TDMI CPU and there aren't any provided libraries to work with. This means that I have to write directly to the registers to make the device display what I want it to. Most of the work is most easily done in C/C++ which are both languages that I am familiar with but want to learn more about. Furthermore, I have a Gameboy Advanced at my disposal so after I finish working on emulators I can play the game on the real thing! My goal is to create a vertical scrolling platformer (similar to doodle jump) for the GBA.

Over the last couple weeks a lot of my time has been spent getting my environment set up and doing readings so I can do basic displays. Because of the small form factor and age of the device, there is not much memory to my disposal so optimization is crucial. Some things are also very slow, I cannot simply write to each pixel as the GBA is simply not fast enough. So far I have learned about bit maps, tile modes, scan lines, vblanks, data alignment, page flipping and much more.

Since the last update it has been a couple more weeks and now I have made some actual progress and I have started development. We can see the first stages of the game above. Currently I am using very basic sprites as I spent much time messing around with sprite editors and had no luck. Maybe I need to spend some more time with the GBA specific editor but I wasn't able to get it to output the data as I wanted. Thus I am creating the sprites manually for now (editing the hex for the pixels as I desire). I am currently focusing on implementing the basics of gameplay most of this is stuff that I was provided when working in Unity. Most recently, I just finished implementing object collision detection. My next step is to actually force the objects to act as rigid bodys and not go through one another and then I will focus on the sliding screen!

PONG - A little break from the GBA

I took a break from the GBA development to work on a new game as apart of a coding challenge. I was given a week to create the classic game of PONG but with the twist of using motion capture so that the players hands control the paddles. After a week of work, I was able to create a fully flushed out game that has multiple game modes, a menu, and a scoring system. The game is fully playable and can be played with 2 players or 1 player against the computer.

The game and a demo video can be found on my Github here

Video game development is a very fun and interesting hobby that I am excited to work on when I have some free time. However, after returning to school for research these projects have been put on hold.

In 2021-2022, 3 classmates and I reached out to some professors at our University to find a physics research project that interested on. We landed on a photonics project with the Quantum Matter Institute. Our sponsor Dr. Jeff Young, wanted us to investigate the theoretical possibility of using silicon photonoic cavities to create a qubit. Thus our job was to do research and model what the cavity would look like, the photon-cavity interactions as well as simulate these interactions.

This project was majorly a theoretical investigation with little coding involved other than the simulations of our theoretical models. A full report of our models and results can be found here. The introduction of this paper serves appeals to anyone with some physics background. However, the rest of the paper is geared to our sponsors and thus is heavily focused on the physics relevant to the project.

In 2020-2021 me and 3 of my classmates took on our first capstone which was to create a machine learning model to help classify an extremely rare kaon decay. This project was done in partnership with TRIUMF and our main contact was at the experiment itself at CERN. Our sponsors were physics researchers so they did not have much background knowledge in machine learning. Thus this was my first self directed machine learning project. Without going too much into the physics, we were only given numerical data. So wanted to first reconstruct and visualize what the detections looked like and see if we could use these visualizations in a CNN. However, there was more data than just those visualizations, so we used a BDT to train off the other data and then combined the two models.

Here we can see the Kaon in green and its decay point indicated by the green dot, then we can see the detected particles afterward indicated by the red line. Right after the decay point we see the STRAW detectors which is what we were trying to visualize given the numerical data. The other detectors present in the experiment were used in the BDT.

In the diagram above you can see a side by side of what the STRAW detectors looked like and what our visualization of the specific straws hit looked like in our CNN. This is what we used to train and we were able to make a decent model with these visualizations. The main goal of this project was not to get an extremely accurate model but to try to isolate which variables have a larger impact on the predicition. With the BDT we were able to do exactly that. The physicsits then took our results and are now researching what this means for the decay!

A full report on the CNN and BDTs used and their results can be found here. This also includes background information on what data was available and a description and explanation of each of the detector parts along with the justifications for the descisons we made. This report was geared towards both our sponsors at TRIUMF/CERN and our instructors so it should be understandble for anyone with some tech/physics background.

In 2020 I entered my first hackathon, NwHacks, with 3 of my closest friends. In just 24 hours we were able to come together and create greenEats, an all in one grocery management app. The app allowed users to input grocery items into the in app inventory using one of three methods - manual input, taking a picture of the reciept, and speech to text. greenEats then also takes in the expiry date and allows users to sort by expiry. Finally the app allows users to select items in their inventory and would reccommend reciepes based on those items. Due to the large number of features we were able to include, greenEats took home the grand prize of first place!

Our devpost shows some more insights into the motivations and technologies used and can be found here

In November 2019, my friend Anthony Ho and I enrolled in a project course that revolved around implementing modern techniques (computer vision and machine learning) in a simulated competition.

The competition takes place in a simulated parking lot environment. This parking lot has 2 loops, and inner and an outer loop, 2 crosswalks with pedestrians crossing at any given time, and a truck driving around the inner loop.

The objective of the competition is to control a robot that goes around this simulated parking lot, and checks if the parked cars have permission to park by reporting all the license plates and spot numbers of the cars. The robot is to do this without hitting the pedestrians or the truck and staying within the lanes, failure to do any of these resulted in point deduction.

A report of our strategy and implementation can be found here.

Hedwig is an autonomous robot that was made to compete in the annual 2nd year Engineering Physics robot competition at UBC. The goal of the competition was to create a completely autonomous robot, that was able to navigate a course to precisely pick up stones off of pillars of different heights and place them into a tray.