Virtual Reaction Game — developing brave new worlds

Posted July 2, 2015 by jomarsevilla
Categories: Gaming, Software, Virtual Reality

You are suddenly riding around Stanford in an Aston Martin convertible.  3D text appears in front of your car with game instructions and a countdown timer.  Ringing bombs spawn all around you; you must locate and defuse them!  If you run out time, you should restart the level… or continue playing amongst colorful exploding bombs which will certainly induce nausea.

This truly describes the premise and experience of Virtual Reaction, an immersive Virtual Reality (VR) game.  My team created this interactive training game to improve a user’s sensory processing and reaction times.  We hypothesize that Virtual Reaction maximizes users’ stimuli and engagement which results in better training and more fun.

arcade

Explaining Virtual Reaction to my friend at the Synapse Arcade. He was the first of only two people to beat the game’s hard level!

New Skills:

  • Unity 5 Game Engine.  Javascript/C# scripts
  • Developed VR environment with Oculus Rift DK2 featuring live-action footage

Next Questions:  During the game’s development, I encountered 4 salient problems that I think epitomizes VR’s biggest challenges:

  1. Lagging Frame Rate:  Our game needed to be more efficient with its computation.  There was noticeable lag when changing your view.
  2. Un-immersive Sound: multi-sensory integration is essential to make our surroundings believable.  Our game featured background music, driving/car/wind sounds, and ringing bombs in 3d space.  When mixed together, the sound wasn’t convincing.
  3. Unreal interactive experience:  without Oculus Touch, input was constrained to computer keys.  Typing does not translate into VR well so users may feel disoriented trying to interact with the game.
  4. VR sickness: the 3 previous issues and several other  factors caused discomfort for users.  Being in VR is always fun, however during its development I required frequent breaks in order to rest my eyes and keep my dinner inside me.

Key Takeaway:  VR will disrupt, enhance, and redefine how we consume media.  Applications are everywhere: gaming, cinema, training, research experiments, surgery, defense, communication, and a lot more.  Right now, it’s a young evolving platform that’s experiencing significant growing pains.  But I believe it’s only a matter of time before people overcome these challenges and unlock VR’s potential.

Screen Shot 2015-07-02 at 2.53.15 AM

Our poster explains the basis of the VR game as a neuroplastic training environment.

Comments: Be the first to comment

Software — it’s icing on the cake!

Posted July 2, 2015 by jomarsevilla
Categories: Software

After completing most EE requirements and electives, I became comfortable with the department’s quirks, smaller classes, familiar classmates.  So it was a shock in my last year to go through the Computer Science introductory sequence.  It was like beginning college all over again – huge classes with hundreds of Freshmen, a new set of rite-of-passage hurdles.  The best assignments were:

  • submitting a blackjack game for a coding competition
  • creating Boggle
  • defusing a ‘bomb’ code within a morass of machine instructions
  • creating a heap allocator, which is the way a computer organizes memory for the user
  • implementing part of a file system
  • managing an army of computer threads to gather an Internet news search. [Might do individual posts on all of these later…]

New Skills:  C systems programming.  C++, Java object-oriented programming.  Python, Linux familiarity.

Key Takeaway:  Building computer hardware is like baking cake.  It’s fun, but it’s more scientific than personal.  It requires expertise because an error can make the cake a disaster.  Figuring out the right mixture of flour, sugar, butter, and eggs.  Balancing flavor extracts or other ingredients for the specific type of batter.  After it is baked (..or fabricated), it belongs to the software guys who decorate the cake and bring it to the party!  First they organize the layers.  Then put fruit, frosting, candles, and write a message based on the occasion (..or application).  Their work is more personal, more creative, closer to the end-user.  It’s quick and messy, but if they screw up, they just wipe it off and start over.

At the party, consumers only see the cake’s presentation, but come to appreciate it fully when eating it.  I don’t yet know where I’ll be in this process, but I know I’ll always love cake.

Comments: Be the first to comment

VisualTouch Music Game — blending HW & SW

Posted July 1, 2015 by jomarsevilla
Categories: Gaming, Hardware, Music, Software

Embedded systems are everywhere.  A digital watch, microwave oven, electronic toy, and your car dashboard are a few disparate examples.  The characteristics of an embedded system are:

  • Made from a combination of hardware and software
  • Task specific. They’re designed to perform one specialized task as opposed to general purpose computers
  • Usually reactive or feedback-oriented.  E.g. most home or kitchen appliances only function when you interact with them.
  • Built with efficiency and frugality.  Your toaster needs a timer and heat control, but it really doesn’t need a full-fledged CPU
  • Should be reliable and stable.  Because they are simple, we expect these machines to work. They shouldn’t require maintenance because it’s often easier and cheaper to buy a new one.

For my required EE Hardware design project, my team developed an embedded system on an Atlys Spartan-6 Development Board.  This board constructs your hardware specification, like an FPGA, but then it can also run C software that you place on top of the hardware.

StanleyFPGA

My brother’s Flat Stanley visited me while I was working in the EE lab.  Sadly, I couldn’t find a photo of our finished project, but this is our Spartan-6 Development Board

We hooked up a webcam, a screen, and speakers to our Board.  We had the screen display the camera’s output.  Our game consisted of these playable modes:

  1. Changes the chord’s duration based on the red/blue/green content of the overall pixel count.
  2. Changes the chord based the amount of movement detected by the camera.
  3. A grid is overlaid on the screen. When a user in front of the camera covers a square of the grid with their hand, the board plays a song.

Skills:

  •  Verilog code in Xilinx Platform Studio configures the processor-based system.  [The hardware platform managed the transfer of data from the camera to the screen.  All the song data was also implemented in hardware.]
  •  C programming in Xilinx Software Development Kit.  [Everything else:  the music player, game controls, game logic, pixel computations were done in software.]

Key Takeaway:    This project was a turning point: creating the hardware infrastructure was important, but the software was the crux of the game! My specialty was more on the hardware side and I realized that I shouldn’t leave college without delving into the software domain.

Screen Shot 2015-07-01 at 8.00.14 PM

DJ mode allows user to mash these pop songs together based on where one’s hand is positioned in the screen’s grid.

Comments: Be the first to comment

Graphics Accelerator Chip

Posted July 1, 2015 by jomarsevilla
Categories: Hardware, Optimization

How often do you watch animated movies, play video games, or run graphical simulations on your computer?  Computer graphics is the science of converting 1’s and 0’s into colored pixels on your screen.  While many applications can do this work entirely on the CPU, high end real-time applications require specialized hardware.

For the final project of EE271: VLSI (Very Large Scale Integration) Systems, we had to increase the performance of a graphics chip to the requested throughput while minimizing chip area and power consumption.  One acceleration technique that we implemented that is easy to visualize is to change the rendering algorithm.  Compare different algorithms in this figure:

Screen Shot 2015-07-01 at 6.05.30 PM

Juan Pineda. 1988. A parallel algorithm for polygon rasterization.

Navigating the stages of this project was very challenging with lots of scary moments and meticulous deliberations. Whenever we attempted to optimize anything, the first step usually broke the system.  But near the end, it felt like a miracle when images rendered properly and quickly.

Screen Shot 2015-07-01 at 7.04.51 PM

One of the images we had to render. He kinda looks like Baymax from the Disney movie.

New Skills:  Graphic acceleration techniques in SystemVerilog and C++

Key Takeaway:  I was inspired when working on this graphics accelerator chip because I felt like I was treading (ok, more like ‘limping’) in the footsteps of my Filipino engineering hero, Dado Banatao.  He started a company around a similar technology.

Comments: Be the first to comment

CPU Optimization — a neverending story

Posted July 1, 2015 by jomarsevilla
Categories: Hardware, Optimization, Software

EE282 Computer Systems Architecture introduced me to more complex CPUs (microprocessors are named Central Processing Units when packaged within complete computers) and to more advanced techniques for architects building CPUs.  There were two big optimization projects:

  1. Optimize Matrix Multiplication:  We had to implement optimization techniques in C to achieve faster matrix multiplication.  A very effective technique is loop unrolling which basically puts more work in each loop iteration to reduce the number of iterations. For example, a for-loop executing a single multiplication needs 100 iterations; the optimized version only need 10 iterations, but each iteration grabs a bigger chunk of data and does 10 multiplications in each loop around.
  2. Optimize Mystery Computer:  There’s a dynamic mystery computer that executes large mystery computations.  We could remotely control the computer’s number of processors, processor speed, and cache system.  We then had to explore paths to an optimal computer configuration among thousands of possibilities.  My partner and I were like Sherlock and Watson; discovering new clues, having eureka moments, and then surmising explanations afterwards.

New Skills:  C programming, performance analysis, and optimization techniques

Key Takeaway:  The official Olympic motto is Citius, Altius, Fortius which is Latin for ‘faster, higher, stronger’.  I think this motto’s underlying sentiment isn’t to celebrate the fastest, highest, strongest but to spur athletes toward continuous improvement.  If CPU development had its own motto it would be faster, smaller, less energy.

Ever since their invention, new CPUs are released that are more powerful, with smaller transistors, and more energy efficient.  And just like there’s no perfect athlete or sports team, there’s no ultimate computer because progress in one area comes at the expense of others.

Comments: Be the first to comment

Microprocessor Design – managing a lifetime of work each second

Posted June 30, 2015 by jomarsevilla
Categories: Hardware

In a series of lab projects, my partner and I completed the design of a pipelined processor and memory system.  No, we didn’t connect every wire in the microchip by hand.  We designed the system with a hardware language that recreates the microprocessor on an FPGA.  [Field-programmable gate array is a microchip with programmable fabric that can be continually redesigned into new microchips.  More on this in a later post…]

The microprocessor chip is the heart of any computer.  It’s the workhorse that grabs data, performs logic on it, and spits out the new data to where it belongs.  One factor that determines computation power is the speed/frequency of its processor.  The computer I’m typing on uses an Intel i5 Processor at 1.8 GHz.  That means every second, it can perform 1.8 billion cycles of computer busywork.

Estimating the average human’s cardiac cycles over a 60-year lifetime: a person will experience (60 heartbeats per minute)*(1440 minutes per day)*(365 days per year)*(60 years) = 1.89 billion heartbeats.  In essence, computers can do great things mostly because humans designed them to manage a lifetime of busywork in each second.

New skills:  Processor design in Verilog code using Xilinx ISE synthesis and analysis tools.

Key Takeaway:  Forget final exam scores; a class should be measured by the number of times a student is stunned with realization, making him see the world a little differently.  For me, this was such a course:  where the perfect blend of lectures, assigned reading, homework problems, and final design project blended perfectly into a world-expanding experience.

Screen Shot 2015-06-30 at 7.33.47 PM

My handwritten diagram of our cache system! Caches store recently used (or spatially close) data for easy-access because the processor is likely to use that data again. Similar to why a browser’s cache will locally store website elements that you commonly visit, so the browser doesn’t have to re-download them over the Internet every time.

Comments: Be the first to comment

Learning Precision

Posted June 30, 2015 by jomarsevilla
Categories: Uncategorized

As much as I enjoyed my electronics, physics, and humanities courses; my favorite class in my first-year…

… was still CALCULUS!

It was a year-long series of courses that taught linear algebra, multivariable calculus, vector calculus, and differential equations.  My absolute favorite topic was vector calculus, where operations and theorems are used to compute 3d vector fields.  This can be visualized as a collection of arrows in space, like an electromagnetic field or a gravitational force.

New skills:

  • Computing with Precision.  Though my calculus tools and intuition have atrophied over time, computing with an attention to detail never goes out of style.
  • Communication with Precision.  I couldn’t get through this course by myself and working with peers late into the night, carefully stepping through pages of integration or proofs forced me to communicate with a precision never-before required.

Next Questions:  Where do these models and techniques manifest themselves in engineering?  [I would soon find out: in many places.  Particularly in an Electromagnetic course that zapped away any nostalgia I still had towards Vector Calculus]

Key Takeaway:  We learned that when a problem seems intractable, usually a change of perspective (…or in our case a change of coordinate systems) is needed.  And after that simplification, you realize you’re just integrating Pringles.

hyperpara

I vividly recall intimidating problems requiring triple integrals to calculate the area and volume associated with hyperbolic paraboloids.

Comments: Be the first to comment

Arduino Music Box

Posted June 30, 2015 by jomarsevilla
Categories: Hardware, Music, Software

In my first year, I took an introductory electronics course that gave us access to

  • analog circuits:  circuits are closed loops featuring electronic components (such as resistors, transistors, LEDs, transistors, amplifiers);  there’s also a power source that raises the voltage, pumps current, putting all the components in action.
  • sensors:  interactive devices (such as buttons, wheels, temperature sensor etc. ) that can sense their surroundings
  • microcrontrollers:  named ‘micro’ because they are small, simple, cheap computers; ‘controller’ because they are mainly are used to run simple logic to control other devices.  Though it can execute code like any computer, it doesn’t have the computing power to run your laptop or phone.

After training us with all these parts and more, we were then let loose to build our own final project.  I had the most fun running simple code on an Arduino Nano microcrontroller.    Using an empty cigar box, I made a music box that plays mp3s from an SD card and displays its song name on a tiny screen.

This was filmed in 2011 when my phone was technologically more similar to a potato than a modern smartphone.

New skills:

  • First time constructing non-trivial circuits and using equations like Ohm’s Law in real life instead in a notebook.
  • C++ programming in using Arduino Integrated Development Environment (IDE) which runs the code.

Followup Questions:  How does the microcontroller work?  How did it interpret the mp3 file and decode into playable sounds? How to organize circuit wirings so it’s not a hot mess?

Key Takeaway:  An auspicious introduction to electronics where I quickly learned that digital domain (running code on microcrontroller) is friendlier than analog domain (soldering little wires and devices together and measuring with oscilloscope).

In retrospect, this key takeaway became a catalyst for a lot of other things.  First impressions DO count!

Comments: Be the first to comment