I am a senior computer science major in the the College of Engineering at Cornell University. My current research interests lie in the areas of human-robot interaction, computer vision, and augmented reality. I am also a member of Cornell's Robotic Personal Assistants Lab run by Prof. Ross Knepper. Currently, I am working on a telepresence robot equipped with a 360 degree RGB-D sensor. The goal is to use the data coming from the sensor to enable the robot to seamlessly and smoothly navigate crowded pedestrian environments. More details about me can be found in my resume.
202 College Ave. #3
Ithaca, NY, 14850
8415 Inverness Way
Chapel Hill, NC, 27516
We present an autonomous system, capable of traversing crowded pedestrian environments, without hindering humans’ paths and complying with socially acceptable standards of motion. The core of this system is a navigation algorithm, designed according to specifications extracted from sociology studies on pedestrian behavior and psychology studies on action interpretation. The algorithm enables the robot to infer humans’ intentions based on their observed trajectories and communicate its own intentions through its own motion. We collectively model the intentions of all pedestrians in the scene as combinations of intended topological routes and intended destinations. Preliminary results present the main components of our system in action. Our approach will be validated with real world experiments involving a robot platform navigating crowded academic spaces.
We created a centralized ROS system and Unity Interface
that allows remote users to control a team of robots. The
users can define what the robots should be doing and utilize
many levels of control. The behavior includes exploration
and the manipulation of objects and moving them to user
defined locations to create structures.
Our centralized system also includes a supplementary
learning module for fine motor tasks. The user can select
to run one of these modules in Baxter’s action graph,
enabling the learning module to take direct torque control
of Baxter’s arm in order to achieve some predefined,
pre-learned objective. The module is pre-trained on a set of
challenging tasks requiring fine-grain physical coordination
such as screwing a bottle cap on, fitting a peg into a
hole, and picking a block off of a tower. Baxter is trained
on these tasks using the Guided Policy Search (GPS)
algorithm developed at Berkeley. The learning module’s
abilities are further enhanced by incorporating a deep spatial
autoencoder to provide the GPS torque net with key feature
points. Using both torque feedback and visual feedback,
Baxter can achieve a high degree of fine-grain coordination,
significantly extending the collective capabilities or our
centralized robot system.
To develop and test the system we used two robots:
Baxter, an industrial robot built by Rethink Robotics with
two 7-DOF arms, and a modified version of a Turtlebot
named Gatlin. Gatlin includes
an 4-DOF PhantomX Pincher arm for manipulation, an
iRobot-Create base for mobility, and a Kinect head mounted
on a 2 DOF turret for 360 degree RGB+D vision.
Computer science and robotics is in an exciting time of rapid progress. Due to advances, robots can perceive and manipulate more objects for low cost and at a higher accuracy. We wanted to create a system that we could use to control the behavior of any robot added to the system. There are many powerful open source algorithms (in ROS) that we can call from our system. 3D interfaces are efficient ways of communicating information and extending some control to users. With practice, people gain dexterity over the interface and information it is conveying. We used the Unity3D Game Engine to create the user interface so we could visualize a 3D representation of the system on any device, in real-time. Unity3D exports to standalone Windows/Linux/Mac clients, iOS, Android, Windows phone, and more, allowing a wide variety of devices for users.
The system can be used as it was intended. We successfully had Gatlin and Baxter following the tasks that the user created in the interface. We were able to make user-defined arrangements of blocks, throw them, and have Gatlin drive routes that were created in the map. We had multiple users controlling the same robots and observing the structures they were building. We have combined many powerful vision and robotic techniques into our system. We are excited to work more on this project. We want to streamline the process of adding robots and their actions to the system. This will allow us to incorporate many of the exciting advances in computer science that will extend what can be perceived and manipulated. These can be put into the system, expanding the capabilities the user has over their environment. The supplementary learning module was never fully incorporated into the overall ROS system due to time constraints. Once the learning module is completed, it’s recommended interface is a ROS messaging service that could define a task to either be strained on or to implement using a trained model. Sending a task would involve sending a cost function to minimize over a set of actions. If the cost function (which might be encapsulated in an enumerated constant) has already been used in a training session, the robot can retrieve the policy it developed in the training session. In this way, the GPS and DSA networks can effectively take control of a given robots actions for a specified task. Training the DSA network has been tested with small amounts of data. However, to actually use this network, we need more data by following the steps in the appendix. Others can train and use this system if they have the data. It can also be extended to include the depth channel.
The camera on the back of the mobile device captures the user's hand pose while the screen displays the calculated
location of the hand overlaid on top of the 3D view from the Kinect on the robot. The mobile device's screen
acts like a window into the robot's perspective. The mobile device's gyroscopes allow precise tracking of orientation, and utilizing the gyroscopes makes the app much more immersive
since when you turn the mobile device, the view rotates accordingly. The arm's end effector would then copy the pose of the user's hand, allowing the user to reach
for an object and grasp it as if the user was standing where the robot is. The pros of this method is that only a mobile device
with a camera is needed to capture hand movements. Thus this will allow the robot controller to be packaged in a app and anyone with a smartphone
could then download it to their phone and control the robot remotely. (security authentication will be added so not just anyone can access the robot)
Although I have just begun to experiment with Unity and ROS, the combination seems to have many possibilities and applications. Here is a short video demo of an initial prototype for the robot's Android controller app, showing a visualization of the RGBD data from the Kinect, joystick controls for moving the robot, and gyro controls for viewing the 3D pointcloud.
The mobile app was created using Unity and the Unity-ROS connection was implemented using the ROS packages "rosbridge" and "web_video_server". Here is a schema for another ROS web interface that we used as a guide during the beginning of this project.
Final class project for ENGRI 1820: Electricity Lights Camera Action: Nanoengineering for the Future of Bits and Bytes.
It is a simple lightweight stress sensor that fits around the wrist
which senses your stress level using heart and galvanic skin response
and indicates it with an LED of different colors. This could certainly
be useful to an ordinary person as a way of tracking everyday stress,
but also has potential medical applications, allowing doctors and patients
to quickly and easily determine the patients stress level, which would
be useful in managing a variety of illnesses from heart disease to anxiety disorders.
It was created by taking advantage of the fact that the sensors and microcontrollers needed already exist and are easily accessible. This allowed us to focus on integrating them into a functioning system rather than worrying about the specifics of each sensor and to devote more time to the problem of accurately interpreting the data and making them work as flexible electronics.
Resources: A tessellated surface that can fit around the wrist, an Arduino microcontroller that can be used to interpret the data, a heart rate or pulse sensor, a galvanic skin response sensor or the materials needed to construct one (a fairly simple operation that mainly requires a conductive surface and an Arduino to regulate the data), multicolored LEDs to indicate your stress level, batteries, and wire.
This project utilizes a program that allows the user to easily collect temperature data from inexpensive thermistor temperature sensors connected to the computer through an Arduino. Contains a Java UI for the Arduino USB serial connection. Converts the voltage through the thermistor into a temperature. Allows the user to set the duration and sampling rate for the data collection. Provides real-time graphs of data... README
This program was included in a research paper entitled "Autonomous Thermal Data Collection" and presented
at the 44th Lunar and Planetary Science Conference (LPSC) in Huston, Texas.
All LPSC 2013 Mars Outreach for North Carolina Students Posters
All of the data we used was pulled from NASA's NeoWs online api available at https://api.nasa.gov/api.html#demo_keyratelimits Each NASA api request returns a JSON file with a list of asteroid objects for a particular week of the year. Each asteroid object returns a diameter, magnitude, velocity, miss distance, approach date, and name. Since each file represents one week of the year, we iterated through all 53 weeks in order to get all of the data for 2016. In our graphs, we only referenced the relative velocities, distances from Earth, and diameters of each asteroid. We chose those criteria because they seemed most relevant for visualizing the potential threat a near-earth object may pose towards people on planet Earth...
"Our client is Ada Chan, an MBA candidate at Cornell. She is the founder of a company that intends to create an online solution to
match part time cleaners with customers (for the purposes of this project we may decide to focus on the Ithaca area). The website we
built is intended to be an alpha version of the company's final product and will mostly be used for product demos to potential
The website's main purpose is to connect customers who need household cleaning services with providers in their local area. The content will reflect this purpose as well as make it as easy to use as possible. We plan to incorporate many features, so it should be very simple to facilitate communication between customers and cleaners. This will ensure that target audience enjoys this service as much as our client does."
This image gallery will be used to host professional pictures of me and my CS projects, so I want to reflect the coding aspect in the look and feel of my design. I used a black background because it adds more contrast to the text and most people associate black backgrounds with computer consoles or terminals. I also wanted the website to feel sleek and modern, just like the iOS and Android operating systems which is why I incorporated large icons and a CSS flip for interaction...
This music library was my first website made using a database and it actually uses a text file to save data. You can filter your music by mood with tags to find a perfect song for any time of day. You can also search by Title, Artist, Album, Year, Rating or a combination. Simply change a search parameter to get a list of matching tracks...
"For this project, our client is a small company called Physics Bus, founded by Erik Herman. Physics Bus is a refurbished old style bus that travels to schools and events as a mobile physics exhibit. The mission of Physics Bus is two fold: first, to spark interest in physics and science in audiences of all ages through fun and interactive exhibits, emphasizing the artistic and creative aspects of physics rather than straight equations; second, the exhibits in the bus are made from also entirely recycled and/or junk materials. The current URL for the Physics Bus site is http://ithacaphysicsfactory.weebly.com/ (physicsbus.org), but our client has expressed his unhappiness with the overall theme, color scheme, and even logo of the site (which is just a slightly modified version of the logo for the parent organization, Physics Factory). What our client was looking for in their site was something that embodies the fun and interactive elements of their project. In particular, our client mentioned that he likes the look of hero images and requested a slideshow on the website's main page. He was also looking for a site that effectively uses social media, blogging, and videos/images to give users a taste of what the Physics Bus is like."
Course information provided by the Courses of Study.
Informal seminar in which current topics in robotics are discussed.
The AI seminar will meet weekly for lectures by graduate students, faculty, and researchers emphasizing work-in-progress and recent results in AI research.
Prerequisite: graduate standing or a previous robotics course such as
MAE 4180 or CS 4758. A background in mathematics is required, especially
linear algebra and probability. A strong programming ability in C++ or
Python is required.
Mobile manipulation is the ability for a robot to interact physically with versatility in the world. As robots become integrated into complex human environments, mobile manipulation is increasingly necessary. Robotic mobile manipulation will enable such applications as personal assistant robots in the home and factory worker in advanced manufacturing. This course covers the fundamental theory, concepts, and systems of mobile manipulation, both software and hardware. It addresses the topics of kinematics, dynamics, controls, grasping, planning, mapping, dealing with uncertainty, and human-robot interaction.
Develops techniques used in the design and analysis of algorithms, with an emphasis on problems arising in computing applications. Example applications are drawn from systems and networks, artificial intelligence, computer vision, data mining, and computational biology. This course covers four major algorithm design techniques (greedy algorithms, divide-and-conquer, dynamic programming, and network flow), undecidability and NP-completeness, and algorithmic techniques for intractable problems (including identification of structured special cases , approximation algorithms, local search heuristics, and online algorithms).
An introduction to machine learning for intelligent systems (e.g. search engines, recommender systems, ad placement). Topics include supervised learning, statistical learning theory, and online learning. Prerequisites: Probability theory (BTRY 3080, ECON 3130, MATH 4710, or strong performance in ENGRD 2700 or equivalent); linear algebra (MATH 1920 or equivalent); CS 2110 or equivalent programming proficiency.
Robotic manipulation is the ability for a robot to interact physically and deliberately in the world. Although long used in factories, robot manipulators will soon appear in home environments as well, helping us with daily tasks. This course covers the theory and concepts involved in programming a robot manipulator, including rigid body mechanics, kinematics, dynamics, path and trajectory planning, control, and stability. At the end of this course, the student is able to program a real manipulator arm to perform autonomous tasks.
Project portion of CS 4700. Topics include knowledge representation systems, search procedures, game-playing, automated reasoning, concept learning, reinforcement learning, neural nets, genetics algorithms, planning, and truth maintenance.
Challenging introduction to the major subareas and current research directions in artificial intelligence. Topics include knowledge representation, heuristic search, problem solving, natural-language processing, game-playing, logic and deduction, planning, and machine learning.
An in-depth introduction to computer vision. The goal of computer vision is to compute properties of our worldâ€”the 3D shape of an environment, the motion of objects, the names of people or thingsâ€”through analysis of digital images or videos. The course covers a range of topics, including 3D reconstruction, image segmentation, object recognition, and vision algorithms for the Internet, as well as key algorithmic and optimization techniques, such as graph cuts and non-linear least squares. This course emphasizes hands-on experience with computer vision, with several large programming projects.
Introduction to the design of systems programs, with emphasis on multiprogrammed operating systems. Topics include concurrency, synchronization, deadlocks, memory management, protection, input-output methods, networking, file systems and security. The impact of network and distributed computing environments on operating systems is also discussed.
Introduction to computer organization, systems programming and the hardware/ software interface. Topics include instruction sets, computer arithmetic, datapath design, data formats, addressing modes, memory hierarchies including caches and virtual memory, I/O devices, bus-based I/O systems, and multicore architectures. Students learn assembly language programming and design a pipelined RISC processor.
Advanced programming course that emphasizes functional programming techniques and data structures. Programming topics include recursive and higher-order procedures, models of programming language evaluation and compilation, type systems, and polymorphism. Data structures and algorithms covered include graph algorithms, balanced trees, memory heaps, and garbage collection. Also covers techniques for analyzing program performance and correctness.
This course covers the mathematics that underlies most of computer science. Topics include mathematical induction; logical proof; propositional and predicate calculus; combinatorics and discrete mathematics; some basic elements of basic probability theory; basic number theory; sets, functions, and relations; graphs; and finite-state machines. These topics are discussed in the context of applications to many areas of computer science, such as the RSA cryptosystem and web searching.
As the next Java course after AP Computer Science, CS 2110 includes review of class structure and recursion as well as an in-depth study of data types. Further along in the course, more advanced topics are taught (e.g. complexity analysis and graph algorithms.)
Programming and problem solving using MATLAB â€“ a language that has great numeric capabilities, as well as excellent plotting and graph functions. Topics included functions, arrays and vectors, iteration, recursion, complexity, dimension, randomness, simulation, the role of approximation, and MATLAB graphics.
Focuses on reading and evaluating the theories and research methodologies used to investigate communication via computer systems. Assignments include student collaborations using electronic conferencing and other advanced communication technologies, as well as reflections on and evaluations of these collaborations in light of current theories and research findings. Topics include virtual teams, videoconferencing, and others as they emerge.
This course provides an introduction into the design of and research on human robot interaction. Students will be working alone and in teams on designing and examining novel ways for robots to interact with people. In parallel to the design projects students will be introduced to current research on human-robot interaction. Topics include: Needfinding, interaction analysis, video prototyping, testing, and systematic evaluation.
As technologies and algorithms that can autonomously take in information, make decisions, and act on those decisions become more and more prevalent, questions arise as to the moral and ethical aspects of their interactions with people and society. What are the philosophical foundations for a new Robot Ethics? Topics covered include social and therapeutic robotics, search and rescue, surveillance, military decision making, financial markets, robot artists, robot scientists, automation, medical robotics, policing, robot smog, and attention dilution.
This course will introduce students to working with data in the context of modern web applications. These include data representation with relational and non-relational databases, data mining to find patterns and make predictions, and graphical presentation for visualization.
The World Wide Web is both a technology and a pervasive and powerful resource in our society and culture. To build functional and effective web sites, students need technical and design skills as well as analytical skills for understanding who is using the web, in what ways they are using it, and for what purposes. In this course, students develop skills in all three of these areas through the use of technologies such as XHTML, Cascading Stylesheets, and PHP. Students study how web sites are deployed and used, usability issues on the web, user-centered design, and methods for visual layout and information architecture. Through the web, this course provides an introduction to the interdisciplinary field of information science.
Our present technology allows us to transmit and display information through a variety of media. To make the most of these media channels, it is important to consider the limitations and abilities of the human observer. The course considers a number of applied aspects of human perception with an emphasis on the display of visual information. Topics include “three-dimensional” display systems, color theory, spatial and temporal limitations of the visual systems, attempts at subliminal communication, and “visual” effects in film and television.
This course provides an introduction to the science of the mind. Most people have privileged access to one mind, yet this internal experience is often misleading and provides little insight into how minds emerge from physical entities like brains (or other substrates). Instead, an objective, deliberated, and multidisciplinary approach is necessary. To that end, cognitive science draws on, and integrates, at least five disciplines: Psychology, Neuroscience, Computer Science, Linguistics, and Philosophy. This course provides an introduction to the insights that these disciplines offer cognitive science while introducing students to current understanding of attention, memory, visual perception, learning, problem solving, language comprehension, and consciousness.
Topics include electrostatics, behavior of matter in electric fields, DC circuits, magnetic fields, Faraday's law, AC circuits, and electromagnetic waves. At the level of University Physics, Vol. 2, by Young and Freedman, 13th ed.
An intro chemistry course specifically for engineers covering basic concepts such as atoms and molecules, the bonding and reactivity of molecules, intermolecular forces in liquids and solids, gases, chemical equilibrium, thermodynamics, quantum mechanics, the periodic table, and kinetics.
Linear algebra and its applications. Topics include matrices, determinants, vector spaces, eigenvalues and eigenvectors, orthogonality and inner product spaces; applications include brief introductions to difference equations, Markov chains, and systems of linear ordinary differential equations. May include computer use in solving problems.
Learned how to apply the concepts of calculus to multiple variable functions. Introduced vector dot products, cross products, and how they can be used in differential/ integral systems. Studied parametric surfaces and volumes in conjunction with double and triple integrals.
Students use a computer to control equipment and acquire measurements in an engineering or scientific laboratory. Topics include data types, analog-to-digital conversion, digital-to-analog conversion, digital in/out, counter/timers, serial communication protocols, error analysis, least-squared fitting, introductory Fourier analysis, electrical waveguides/transmission lines, stepper motors, and introductory optical microscopy. Laboratory activities include practice exercises, measurement of pulse reflections in a transmission line, and development of a laser scanning microscope system. Data analysis is performed primarily using MatLabTM and instrument interfacing is primarily using LabVIEWTM, but also C++. Students develop effective written communication skills in the context of science and engineering. They prepare progress reports, technical reports, and formal articles based on laboratory work. Outcome 1: Learn techniques for interfacing computers with engineering and scientific measurements. Outcome 2: Extend programming skills to using I/O operations in C++, graphical programming, and “virtual instrument” concepts. Outcome 3: Improve technical writing skills in various forms, progress reports, technical reports and rapid, brief communications.
Hands on course with many labs, each devoted to the study of a common electronic component (e.g. Vacuum Tube Amplifiers, Transistors). Fabricated our own semiconductor devices (Solar Cells) in the nano lab and studied other forms of solid state electronics. In addition, there was also a final class project where my partner and I created a wearable lie detector glove with a Lilypad Arduino and galvanic skin response.
Provides an introduction to statistical inference and to principles of probability. It includes descriptive statistics, principles of probability, discrete and continuous distributions, and hypothesis testing (of sample means, proportions, variance). Regression analysis and correlation are introduced.
Analysis of aggregate economic activity in relation to the level, stability, and growth of national income. Topics may include the determination and effects of unemployment, inflation, balance of payments, deficits, and economic development, and how these may be influenced by monetary, fiscal, and other policies.
From the beginning of the twentieth century to the present moment, movies – and in particular Hollywood – have profoundly influenced the ways in which people see, think and talk about the world. Focusing mostly on Hollywood film, this course introduces the study of American cinema from multiple perspectives: as an economy and mode of production; as an art form that produces particular aesthetic styles; as a cultural institution that comments on contemporary issues and allows people to socialize. We will consider the rise of Hollywood in the age of mass production; the star system; the introduction of sound and the function of the soundtrack; Hollywood’s rivalry with television; censorship; the rise of independent film, etc. Weekly screenings introduce major American genres (e.g. science fiction, film noir, the musical) and directors (e.g. Hitchcock, Kubrick, Tarantino).
ENGL 2890 offers guidance and an audience for students who wish
to gain skill in expository writing—a common term for critical,
reflective, investigative, and creative nonfiction. Each section
provides a context for writing defined by a form of exposition,
a disciplinary area, a practice, or a topic intimately related
to the written medium. Course members will read in relevant published
material and write and revise their own work regularly, while
reviewing and responding to one another's. Students and instructors
will confer individually throughout the term. Topics differ for
SEM 103 - Topic: Legal Science Fictions
Science fiction writers build whole new social systems, and questions of how to govern these new societies inevitably come up. Ought this robot be considered a legal person? Does this cool new policing tactic infringe our rights? Should earth laws apply in space? In this course, we'll consider how such legal topics as personhood, equality, and criminality arise in science fiction and in real cases, and how issues of gender, race, labor, and policing and punishment are complicated by technology in our own world.
Explores the relationship between acts of public protest and those created for the theatre. An in-depth study of how the oppressed express themselves through writing and acting. Through in-class discussions and peer review, we honed our critical and persuasive writing while developing exact, articulate, and well-structured arguments.