Ballon Ninja

Introduction

Welcome to "Balloon Ninja," a dynamic and interactive game that illustrates the practical applications of embedded systems. Built with the Raspberry Pi 4, this game not only provides entertainment but also serves as a testament to the power of integrated hardware and software. It utilizes real-time processing and color detection to create an immersive experience, where players slice through virtual water balloons with the swipe of a finger or the motion of a colored object.

Developed as part of an embedded systems curriculum, "Balloon Ninja" exemplifies how multithreading and computer vision can be harnessed in a playful yet complex application. This project invites players to experience firsthand how embedded technology can be applied in gaming, providing a glimpse into its real-world utility.

Project Objective:

The project aims to demonstrate the capabilities of embedded systems by developing an interactive game that employs multithreading for efficient real-time processing and OpenCV for dynamic color tracking. The game, designed for the Raspberry Pi platform, showcases the integration of hardware and software to create an engaging user experience that responds instantly to physical interactions.

Design

hardware

We connected a Raspberry Pi Camera with the Raspberry Pi 4 through the camera port and use a cardboard to stablize the camera.

Initial Concept and UI Design:

In the "Initial Concept and UI Design" phase of "Balloon Ninja," we focused on developing a user interface that seamlessly transitions between different game levels. This stage involved creating a layout adaptable for diverse screen sizes, ensuring a consistent and intuitive user experience. Special attention was given to the unique display characteristics of piTFT screens and standard monitors, balancing functionality and aesthetics. The objective was to establish a solid foundation for user interaction, setting the stage for the subsequent integration of game mechanics. An early UI design is shown below.

Game Mechanics Development:

In this phase of "Balloon Ninja," we meticulously built the gameplay in stages. Starting with generating and moving a single type of balloon, we focused on ensuring smooth motion and interaction. Next, we introduced the point-click popping mechanism, fine-tuning the response to user inputs. Subsequently, the slicing cut feature was added, creating a more dynamic interaction. The game's complexity further increased with the addition of different balloon types, each with unique behaviors and scoring. This incremental approach ensured each game element was robust before integrating it into the complete game system. Further enhancing the game's challenge, we adjusted the difficulty levels by modifying the balloon generation rate. This was especially crucial for higher difficulty settings where an increased number of balloons, including point-deducing red bombs, required players to be more selective and strategic in their interactions. Further enhancing the game's challenge, we adjusted the difficulty levels by modifying the balloon generation rate. This was especially crucial for higher difficulty settings where an increased number of balloons, including point-deducing red bombs, required players to be more selective and strategic in their interactions. Additionally, our testing revealed that the immediate disappearance of balloons upon popping was visually jarring. To create a more polished and engaging visual experience, we implemented a fading animation for the balloons. This subtle change meant that balloons would gradually disappear from the screen, adding a touch of realism and visual flair to the game. This enhancement significantly improved the overall aesthetic of the game, making the interactions feel more fluid and natural.

Integration of Game and UI:

We harmonized the developed game mechanics with the user interface. This crucial stage involved ensuring the game elements, such as balloon popping and slicing, were seamlessly woven into the UI structure we had created. Special attention was paid to transitions between different game levels, aiming for fluidity and intuitiveness. This integration was key in creating a cohesive gaming experience, where the mechanics and UI complemented each other, enhancing the overall interactivity and playability of the game.

Color Tracking Development

We developed three distinct prototypes for color tracking, each leveraging OpenCV. To determine the most efficient prototype, we used `htop` for real-time CPU usage monitoring. The final choice was made not only based on performance but also its ability to accurately detect a specific light blue color. We developed a specialized color range testing code that displayed masks and results for different HSV ranges, enabling us to fine-tune and select the optimum range. This testing was crucial for minimizing background interference. After selecting the best prototype, we encapsulated its functionality into the `ColorTracker` class, significantly improving the integration process with the game's UI and mechanics. This step was vital in enhancing the game by allowing players to interact through physical motion, bringing a unique and immersive dimension to the gameplay experience.

Final Integration

We meticulously combined the game mechanics, UI, and the `ColorTracker` class into a unified whole. This integration was challenging, particularly in ensuring that the color tracking functionality seamlessly meshed with the game's core mechanics and interface. A significant challenge was managing the amplification factor, which was crucial for adapting the game to different screen resolutions while maintaining consistent gameplay and visuals. We thoroughly tested and refined this integration, focusing on optimizing responsiveness and player engagement. This final step was vital in crafting a cohesive and immersive gaming experience, effectively demonstrating the synergy of embedded systems and advanced computer vision techniques in interactive gaming.

Test

Black-Box Testing of Game Mechanics:

The Black-Box Testing involved an in-depth evaluation of the game's various components, as developed in the earlier design stages. We systematically tested each aspect - from the interactive UI, ensuring user-friendly navigation and adaptability across different screens, to the core game mechanics like balloon popping. The testing here was crucial to ensure that the individual elements not only worked in isolation but also harmoniously when integrated. This phase was about validating the design choices made in the earlier stages, confirming the functionality and user experience before moving to more complex aspects like latency reduction and color tracking integration.

Latency Reduction and Processing Optimization:

A key challenge encountered was a noticeable lag in the game's response to player actions, particularly in slicing detection. To address this, we experimented with various camera resolutions, ultimately finding that a lower resolution significantly improved processing speed. This optimization was crucial for maintaining the fluidity of the game, ensuring that player gestures were translated into game actions with minimal delay. This phase was iterative, involving repeated testing and adjustments to strike the right balance between visual quality and real-time responsiveness.

Balloon Popping Mechanism Refinement:

We delved deep into enhancing the balloon interaction. Initially, the game encountered errors with single-point clicks and rapid successive clicks, which caused unexpected behavior. We introduced conditional statements to resolve these issues, but new challenges emerged with different clicking patterns and the combination of point clicks and slicing gestures. To address these, we designed a dedicated point clicking popping function. This allowed us to separate the point click pop and slice cutting actions, ensuring they operated in parallel without causing bugs. This refinement was critical for a smooth, bug-free gaming experience.

Motion Physics and User Experience Enhancement:

Considerable attention was given to refining the dynamics of balloon movement and enhancing visual elements for a more immersive experience. Key to this was the division of the screen into quarters, ensuring varied trajectories for the balloons. This segmentation allowed for more strategic gameplay, as balloons in the side quarters were programmed to move in specific directions, while those in the middle followed random paths. This design choice not only kept balloons on screen longer, giving players ample reaction time, but also introduced a layer of strategy in managing different balloon behaviors.

Result

The game demonstrated effective OpenCV integration, successfully detecting light blue colors while eliminating background interference. Despite a residual delay, it did not impact the gameplay experience. The UI was highly praised for its aesthetics and smooth level transitions. Players particularly enjoyed the fading animation and the slicing path's visual appeal, lasting 0.2 seconds. Meeting our primary goal, we successfully blended OpenCV with a fruit ninja-like game, akin to motion detection in Nintendo games. This project enhanced our skills in Python class creation, OpenCV application, and CPU usage monitoring to optimize performance.

Conclusion

The conclusion of "Balloon Ninja" is a reflection on our comprehensive journey in embedded systems and game design. Embarking on this project, we aimed to seamlessly blend OpenCV technology with an interactive gaming experience, drawing inspiration from console gaming dynamics. The venture exceeded our expectations, not only in achieving a harmonious integration of advanced technologies but also in the richness of learning it offered. We delved deep into the intricacies of Python programming, real-time image processing, and efficient resource management, gaining invaluable insights. The positive reception of the game's design and functionality affirmed the potential of merging traditional gaming principles with cutting-edge computer vision techniques. This experience has significantly expanded our skill set and perspective, underscoring the immense possibilities that lie in the intersection of technology and entertainment. As we conclude this project, we are left with a strengthened foundation and an inspired outlook towards future innovations in interactive gaming and embedded system applications.

Future Work

For future work on "Balloon Ninja," we aim to explore more advanced OpenCV features to enhance game interaction, like recognizing hand motions and gestures so that no blue color objects are needed. We also plan to optimize game performance for higher resolutions while maintaining low latency. Integrating multiplayer capabilities and developing more complex balloon patterns and game levels will further enrich the user experience. Additionally, leveraging machine learning for more intuitive and responsive gameplay is on our roadmap.

Parts List

• Raspberry Pi 4 $35.00
• Raspberry Pi Camera V2 $25.00

Total: $60

References

PiCamera Document
PiCamera Datasheet
Pygame Document
Previous Project (ASL Learning Game)
Color Detection
Hand Contour Detection

Code

This is the github repository all the codes:

Final Project Code