Introduction
We aim to integrate AR technology into warehouse systems, leveraging its potential as a groundbreaking human-machine interface. Our focus is on developing a multiuser facility design software that seamlessly blends virtual objects with the physical world
Premise
We have an existing warehouse system with a conveyor and rack for two different products. To optimize storage, we plan to add a new rack. Our system involves using a virtual rack for simulating the storage process. Before the simulation, we adjusted various factors and the new rack's position to find the best combination.
The system engages two users – an operator transferring products and a manager overseeing the layout. Both utilize HoloLens 2. The manager initiates the simulation, while the operator transports products using a cart. This setup allows us to simulate the impact of human factors on the system.
Demo Video
Overview
Project basic settings
Platform
HoloLens 2
Tool Used
Unity, Photon, MRTK, Vuforia
Team
3 Members
Duration
5 Months
My Role
Programmer, Designer
My Responsibility
What I did in the project
Facility Planning and Simulation in MR is a 5-month project with a team of 3 members. I serve as both the programmer and designer, while the other two members handle data analysis.
System Process
How does the process work
Choose Role
In the first scene, both the operator and the manager select their roles. In the second scene, connected through Photon Network, they share the same virtual space. It's crucial for both users to start in the same room position to maintain scene consistency. The manager can also opt for 'Manager Only' mode when there's no operator, allowing them to handle layout planning and product transfer tasks.
.png)
Manager
Manager Preparation
The manager uses image tracking to position existing equipment and introduce the virtual rack into the scene. An AR ruler aids in precision during layout planning. Following this, the manager configures simulation parameters via the manager panel. Once preparations are complete, the manager informs the operator and initiates the simulation.
Manager Supervision
The manager monitors the simulation and real-time assessments. If issues such as excessive errors or poor performance arise, the manager can promptly halt the simulation to make necessary adjustments to factors or equipment positioning.
View Assessment
After each simulation, the manager assesses and compares the results with previous runs. Over multiple iterations and adjustments, the manager optimizes the system, taking into account human factors introduced by the operator and the assessments.
Operator
Operator Waiting
The operator must wait for the manager to start the simulation.
Operator Transferring
The operator's task is to move products to the cart and transfer them to the respective racks within the time set by the manager. An image target on the cart tracks its position. If the system loses track of the image, it automatically adjusts the cart's position relative to the operator.
Scenario
Test Scenarios
Assessment Result
Conclusion
To address industrial demands, our system developed an augmented reality environment incorporating real and virtual equipment. It assists managers in facility planning, simulates processes, and provides assessments across both real and virtual realms. By integrating multiple HoloLens2 devices and incorporating a human operator, we effectively replicated the influence of human factors within a warehouse system – a feat not commonly achieved by most software.
Further Improvements
Improvements after the semester ended
More Realistic Hand Gesture
We enhanced the operator's hand gesture for loading and unloading products, shifting from tapping to a more precise process of grabbing and placing items. This improvement closely mimics the real human transferring process.
Improvement Suggestion
.png)
The system not only computes assessments but also offers recommendations for factor adjustments based on the results and manager-defined thresholds, aiding in warehouse system optimization.
Operator Feedback
During the simulation, the operator can provide feedback to the manager through the system, reporting issues like queue overflows or human errors. The manager can then assess whether it's necessary to halt the simulation.
Origin Point Adjustment
.png)
If the origin point of either user shifts, affecting object positions, both users can reset the coordinates by using the 'Reset Origin' button on their menus, while remaining at the entry point of the scene.