top of page

A quick way to browse



System Process



Facility Planning and Simulation in MR


We aim to integrate AR technology into warehouse systems, leveraging its potential as a groundbreaking human-machine interface. Our focus is on developing a multiuser facility design software that seamlessly blends virtual objects with the physical world


We have an existing warehouse system with a conveyor and rack for two different products. To optimize storage, we plan to add a new rack. Our system involves using a virtual rack for simulating the storage process. Before the simulation, we adjusted various factors and the new rack's position to find the best combination.

The system engages two users – an operator transferring products and a manager overseeing the layout. Both utilize HoloLens 2. The manager initiates the simulation, while the operator transports products using a cart. This setup allows us to simulate the impact of human factors on the system.

Warehouse Shelves from Above

Demo Video



Project basic settings


HoloLens 2

Tool Used

Unity, Photon, MRTK, Vuforia


3 Members


5 Months

My Role

Programmer, Designer

My Responsibility

What I did in the project

Facility Planning and Simulation in MR is a 5-month project with a team of 3 members. I serve as both the programmer and designer, while the other two members handle data analysis.

System Process

How does the process work

Choose Role

In the first scene, both the operator and the manager select their roles. In the second scene, connected through Photon Network, they share the same virtual space. It's crucial for both users to start in the same room position to maintain scene consistency. The manager can also opt for 'Manager Only' mode when there's no operator, allowing them to handle layout planning and product transfer tasks.

螢幕擷取畫面 (149).png


Manager Preparation

The manager uses image tracking to position existing equipment and introduce the virtual rack into the scene. An AR ruler aids in precision during layout planning. Following this, the manager configures simulation parameters via the manager panel. Once preparations are complete, the manager informs the operator and initiates the simulation.

Manager Supervision

The manager monitors the simulation and real-time assessments. If issues such as excessive errors or poor performance arise, the manager can promptly halt the simulation to make necessary adjustments to factors or equipment positioning.

View Assessment

After each simulation, the manager assesses and compares the results with previous runs. Over multiple iterations and adjustments, the manager optimizes the system, taking into account human factors introduced by the operator and the assessments.


Operator Waiting

The operator must wait for the manager to start the simulation.


Operator Transferring

The operator's task is to move products to the cart and transfer them to the respective racks within the time set by the manager. An image target on the cart tracks its position. If the system loses track of the image, it automatically adjusts the cart's position relative to the operator.


Test Scenarios

Assessment Result



To address industrial demands, our system developed an augmented reality environment incorporating real and virtual equipment. It assists managers in facility planning, simulates processes, and provides assessments across both real and virtual realms. By integrating multiple HoloLens2 devices and incorporating a human operator, we effectively replicated the influence of human factors within a warehouse system – a feat not commonly achieved by most software.

Further Improvements

Improvements after the semester ended

More Realistic Hand Gesture

We enhanced the operator's hand gesture for loading and unloading products, shifting from tapping to a more precise process of grabbing and placing items. This improvement closely mimics the real human transferring process.

Improvement Suggestion

螢幕擷取畫面 (150).png

The system not only computes assessments but also offers recommendations for factor adjustments based on the results and manager-defined thresholds, aiding in warehouse system optimization.

Operator Feedback


During the simulation, the operator can provide feedback to the manager through the system, reporting issues like queue overflows or human errors. The manager can then assess whether it's necessary to halt the simulation.

Origin Point Adjustment

螢幕擷取畫面 (147).png

If the origin point of either user shifts, affecting object positions, both users can reset the coordinates by using the 'Reset Origin' button on their menus, while remaining at the entry point of the scene.

bottom of page