Paul DeCarlo, Kinect4NES (Yes, Kinect to a Classic NES), is back and continues to push forward on his cool project, Kinect4NES...
Training Kinect4NES to Control Mike Tyson’s Punch-Out!
In a previous post, I talked about how to create an interface to send controller commands to an NES based on interaction with the Kinect v2. The idea was successful, but I received a bit of feedback on the control being less than optimal and a suggestion that it would likely work well with a game like Mike Tyson’s Punch-Out.
This aroused an interesting challenge, could I create a control mechanism that could allow me to play Mike Tyson’s Punch-Out using Kinect4NES with enough stability to accurately beat the first couple characters?
Let’s first look at how control was achieved in the first iteration of Kinect4NES. There are essentially 2 ways of reacting to input on the Kinect, using a heuristic-based approach based on relatively inexpensive positional comparison of tracked joints or gesture based tracking (either discrete or continuous). For my initial proof of concept, I used the following heuristic-based approach:
...
From here, I incorporated the relevant bits into GestureDetector.cs. In my original implementation, I iterated through all recorded gestures and employed a switch to perform the button press when one was detected. This proved to be ineffecient and created inconsistent button presses. I improved this significantly in my second update using a dictionary to hold a series of Actions (anonymous functions that return void) and a parallel foreach, allowing me to eliminate cyclomatic complexity in the previous switch while allowing me to process all potential gestures in parallel. I also created a Press method for simulating presses. This allowed me to send in any combination of buttons to perform behaviors like HeadBlow_Right (UP + A). I also implemented a Hold method to make it possible to perform the duck behavior (press down, hold down). In the final tweak, I implemented a method to produce a RapidPress for the Recover gesture. This allowed me to reproduce a well known tip in Punch-Out where you can regain health in between matches by rapidly pressing select.
This was a rather interesting programming excercise, imagine coding at 2 in the morning with the goal of optimizing code for the intent of knocking out Glass Joe in a stable repeatable manner. The end result wound up working well enough to where a ‘seasoned’ player can actually TKO the first two characters with relative regularity. In the video at the top of this post, the player had actually never used the Kinect4NES and TKO’d Glass Joe on his first try. As a result, I am satisfied with this experiment, it was certainly a fun project that allowed me to become more familiar with programming for the Kinect while also having the joy of merging modern technology with the classic NES. For those interested in replicating, you can find the source code on github. If you have any ideas on future games that you would like to see controlled with Kinect4NES, please let me know in the comments!
Project Information URL: http://pjdecarlo.com/2015/02/training-kinect4nes-to-control-mike-tysons-punch-out.html
Project Source URL: https://github.com/toolboc/Kinect4NES
Contact Information:
Follow @CH9
Follow @Coding4Fun
Follow @KinectWindows
Follow @gduncan411
Continue reading...
Training Kinect4NES to Control Mike Tyson’s Punch-Out!
In a previous post, I talked about how to create an interface to send controller commands to an NES based on interaction with the Kinect v2. The idea was successful, but I received a bit of feedback on the control being less than optimal and a suggestion that it would likely work well with a game like Mike Tyson’s Punch-Out.
This aroused an interesting challenge, could I create a control mechanism that could allow me to play Mike Tyson’s Punch-Out using Kinect4NES with enough stability to accurately beat the first couple characters?
Let’s first look at how control was achieved in the first iteration of Kinect4NES. There are essentially 2 ways of reacting to input on the Kinect, using a heuristic-based approach based on relatively inexpensive positional comparison of tracked joints or gesture based tracking (either discrete or continuous). For my initial proof of concept, I used the following heuristic-based approach:
...
From here, I incorporated the relevant bits into GestureDetector.cs. In my original implementation, I iterated through all recorded gestures and employed a switch to perform the button press when one was detected. This proved to be ineffecient and created inconsistent button presses. I improved this significantly in my second update using a dictionary to hold a series of Actions (anonymous functions that return void) and a parallel foreach, allowing me to eliminate cyclomatic complexity in the previous switch while allowing me to process all potential gestures in parallel. I also created a Press method for simulating presses. This allowed me to send in any combination of buttons to perform behaviors like HeadBlow_Right (UP + A). I also implemented a Hold method to make it possible to perform the duck behavior (press down, hold down). In the final tweak, I implemented a method to produce a RapidPress for the Recover gesture. This allowed me to reproduce a well known tip in Punch-Out where you can regain health in between matches by rapidly pressing select.
This was a rather interesting programming excercise, imagine coding at 2 in the morning with the goal of optimizing code for the intent of knocking out Glass Joe in a stable repeatable manner. The end result wound up working well enough to where a ‘seasoned’ player can actually TKO the first two characters with relative regularity. In the video at the top of this post, the player had actually never used the Kinect4NES and TKO’d Glass Joe on his first try. As a result, I am satisfied with this experiment, it was certainly a fun project that allowed me to become more familiar with programming for the Kinect while also having the joy of merging modern technology with the classic NES. For those interested in replicating, you can find the source code on github. If you have any ideas on future games that you would like to see controlled with Kinect4NES, please let me know in the comments!
Project Information URL: http://pjdecarlo.com/2015/02/training-kinect4nes-to-control-mike-tysons-punch-out.html
Project Source URL: https://github.com/toolboc/Kinect4NES
Contact Information:
Follow @CH9
Follow @Coding4Fun
Follow @KinectWindows
Follow @gduncan411
Continue reading...
