KITECH: Building a factory digital twin and synthetic data pipeline for manufacturing AI

In this recorded presentation, Dr. Hongin Won, who leads the Manufacturing AI Collaboration Team at the Korea Institute of Industrial Technology (KITECH) and Woojin Park, Technical Account Manager at Unity Korea, walk through a 14-week project that turned a casting plant into a digital twin. They cover how the team built the scene from point cloud data with no CAD using Unity Industry, how they used Unity AI to accelerate optimization and UI work, and how they turned the twin into a synthetic data pipeline that trains AI on human-robot collaboration.

What you will learn

• How the team built a metal casting foundry digital twin was built from point cloud data, with no CAD source files
• Where Unity AI, Asset Manager, Asset Transformer, and Version Control fit into a production pipeline
• How to ground a digital twin against real-world measurements, and calibrate the gaps that remain
• How to generate labeled training data automatically instead of annotating frames by hand

This presentation was recorded at the Unite Seoul conference in July 2026.

Video transcript

Speakers

• Woojin Park, Technical Account Manager, Unity Korea
• Dr. Hongin Won, Lead, Manufacturing AI Collaboration Team, Manufacturing AI Research Center, Korea Institute of Industrial Technology (KITECH)
• Jaehoon Hwang, Researcher, KITECH

Runtime: 41 minutes

About this transcript: This transcript is edited for reading.

Introduction: a skeletal factory scene that became a living digital twin

[00:00] Woojin Park: Today, I will be talking about a Unity digital twin-based synthetic data pipeline for manufacturing AI. I am Woojin Park, a Technical Account Manager at Unity Korea. At Unity, I am responsible for technical support for industrial customers. In this project, I will be presenting what we did over 14 weeks with KITECH to support building a factory digital twin.

If I were to sum up today's presentation in one sentence, it would be this. It is the journey of a skeletal factory scene that became a living digital twin.

[00:51] I will be talking about the journey that the Korea Institute of Industrial Technology and Unity took using Unity AI. I will first introduce Unity AI and the related products. Then I will turn to the project itself, KITECH VPH-Metal.

Before we get started, let me show you the final result video. The area you are seeing now is where the molding machine makes the sand molds. Here the cast products have their sprues removed, pass through the degating area, and move on to the final grinding and post-processing area. By the time this presentation is over, I think you will understand how this scene was built in Unity.

[01:38] I have sat through quite a few sessions and seminars on physical AI and related topics. Usually in those sessions you see photos, images, or videos of factories like this, and you think, "Okay, I get that you can build a digital twin with Unity or another tool." But how many people it really takes, how much time to plan for, and what tools can speed it up? Honestly, that kind of information is usually hard to come by. So today I will talk about which products we used and how we built it, as well as how we were able to accelerate the process with AI.

What Unity AI includes: Agent, MCP Server, and Generators

[02:23] Unity AI is made up of the following. The first is Unity Agent. Then there is MCP Server, and also Generators. For now, it is enough to keep these key terms in mind.

Unity Agent lets you use AI based on Claude, for example, or Gemini, directly inside the Editor. As for MCP Server, a great many environments in Korea run on closed networks. So if your company has an agent like Claude or Codex set up internally, you can connect it through the protocol and use it with Unity.

So what is the biggest difference between Unity Agent and MCP Server? Unity Agent runs the skills we built ourselves, some 70 to 80 of them, on the back end. So it can accelerate Unity development that much further.

[03:12] What Unity is ultimately aiming for is 3D real-time applications. When you think about a 3D real-time application, you need animation, you need sound, you need objects, and you need lots of textures and images. So it is not just about writing code. If you use Unity Agent or MCP Server, you can also use the AI assets the Generators produce for each asset type.

Unity Industry tools: Asset Manager, Version Control, and Asset Transformer

[03:45] In addition, Unity Industry includes Asset Manager, a tool for managing assets, Version Control, a tool for version management, and Asset Transformer, which gets your assets or CAD models ready for direct use in simulation. Those are the three tools.

Starting without CAD: from point cloud to mesh

[04:15] Having seen the finished version of today's project earlier, many of you are probably wondering where it all began. In projects like this, people who have CAD data usually bring that CAD into Unity and proceed with the project. But we had no CAD to work from, so we started from point cloud data.

We then converted that point cloud into simplified low-poly meshes, and after that, using Asset Transformer, Asset Manager, Unity Version Control, Unity AI, and a number of other tools and packages, we carried out the project over 14 weeks. What you are seeing now is the point cloud visualized in Unity from the same angle.

[05:04] On the left you have the assets, and then the different tools, such as Asset Transformer, Asset Manager, automation, Editor, and AI. I will explain where each of these tools was used. The part you are probably most interested in is the training section on the far right, or the simulation and digital twin part. Ultimately, I will explain how the point cloud was connected to Unity's Asset Manager and to outputs such as the digital twin.

Asset Transformer: optimizing assets and making them simulation-ready

[05:42] Let's start with Asset Transformer and Unity AI. The clothing on the left is changing slightly. In the lower right, you can see the same clothing rendered at 11,000 polygons and at 1.1 million, and how the polygon shapes differ in each case.

Asset Transformer is a tool that, when you have a CAD file or an object file, lets you easily optimize and lighten it. With the clothing on the left, there is almost no visible difference to the naked eye. So Asset Transformer makes the model as light as possible for the computer to render, while keeping any visible difference to a minimum.

[06:38] If you look at the cubes rotating at the bottom, even when rotating cubes with the same code, the white cube rotates right around its own center, while the yellow cube looks like it is rotating around some other point. In CAD modeling, this maps to the concept of an origin. Depending on whether that origin is set to what we call the center of mass, the center of the bounding box, or just some random value, even if you write the code properly, the result can turn out completely differently. So these parts had to be corrected and improved, and we called this process "simulation-ready."

[07:26] At Unity, we used Asset Transformer and worked together with KITECH on how to perform optimization. Not every task worked perfectly. In the case of the ladder, for example, when moving the pivot, there was an issue where it got compressed into a straight line. In this case, we used Unity AI to analyze the actual geometric cause, and then created a skill set that could be applied across the entire project. So we wrote skill markdowns and applied them to the entire factory scene at KITECH, turned static objects like these into objects that could move, then carried out the first round of optimization work.

Managing versions in Asset Manager: 627 source files down to 218

[08:25] As we worked through the project, what we first showed you was the point cloud data, then the low-poly data that was generated by turning it into a mesh, and the simulation-ready data we just showed, which had been optimized and lightened. So there were these three versions of the data. But to be honest, there were probably many more versions than that. To avoid getting confused by all these versions, and to pull the data we needed at the right moment, version control was essential.

[08:45] So the tool we used was Asset Manager. With CAD files, previews are normally not supported. CAD has plenty of advantages, but 3D objects are heavy to render, and without previews, it can be hard to tell which asset is the one you need.

[09:03] Asset Manager provides previews for every object. You can download them directly in the Editor, or upload them and use them right away. It also includes features for converting into other formats, or automatically optimizing them.

Rather than simply uploading each model file one by one, we defined a set of rules and created ontology-based AAS data, then uploaded it to Asset Manager. We uploaded in a range of formats, not just the general-purpose FBX but USD as well, and other formats as needed. So we originally had a total of 627 source files, and after applying a single standard, we reduced them down to 218 files and uploaded them.

Unity Version Control across URP, HDRP, and USD

[10:07] The next part we worked on was version control. For many of you, version control probably brings Git to mind. But with Git, it is basically very difficult to manage images or large files.

In our case, we ran this point cloud in Unity's URP render pipeline, and we also ran it in the higher-fidelity HDRP. So to maintain both versions and add features, we used Unity Version Control. Over the 14-week period, we separated branches for URP, HDRP, and USD, kept the Editor under integrated configuration management, and handled 77 versions based on changesets.

Building the runtime dashboard with Unity AI

[11:18] Now that the assets were ready and version control was in place, it was time to move into real development.

When people first say they want a digital twin, the first thing they ask for is a dashboard. In the past, building this dashboard meant bringing in UI and UX designers, and building each feature out neatly as a class, and so on. That was what it used to take.

[11:32] But now, if you make an image of the data you want to show in this factory, or make a concept image, the skills inside Unity AI run in the back end, and it turns that straight into an interactive UI dashboard.

Usually, in the field, there are many cases where PLC data cannot be connected directly. In our case as well, there were firewall and security-related issues, so we worked with the researchers and created a kind of dummy PLC dataset, then connected it to the dashboard inside Unity. We carried that whole process through together.

Custom Editor tools for the magnet scenario

[12:21] What we worked on next was not just a visually convincing simulation, but a genuinely fact-based project. After creating the dashboard at runtime, we also needed to create a dashboard or a custom editor inside the Editor itself.

In this factory, the first scenario involved a magnet moving and using magnetic force to pick up objects. To simulate how much magnetic force is needed for pieces of metal to be picked up or not picked up, instead of hard-coding each value, we exposed them directly in the Editor. So things like each cart's status and the magnet's status, and even how the physical simulation behaves, were all developed using Unity AI.

Creating the furnace effect from a concept image

[13:21] The next thing I will talk about is the furnace effect. Some of you may be thinking you need visual effects.

One of the most powerful features of Unity AI and MCP is that they can capture the Scene View or Game View. When you are working on a project, just as a phone applies color correction, colors can look different inside the Unity Editor or Game View. To create the impact you want within those colors, you need things like color combinations. In the past, that was an area artists had to handle.

[14:01] But now you can tell the AI, "Create four spheres in the scene, apply a furnace effect to them, and keep updating it until it looks closest to the concept or concept image I give you." Unity AI then finds and creates the most suitable effect, and even applies it to the actual scene all in one go.

Converting URP to HDRP

[14:24] After that, once we had created the URP version, we converted it to HDRP. During the process of converting URP to HDRP, I was busy with something else, and I gave the AI the graphics settings, concepts, and other details I had prepared. The footage you see here is the result of the AI improving them step by step over time to match that concept image.

[14:51] At first it was either a black screen or an overly bright one. Then it went back to the original, got a bit brighter, and went through these steps on its own, so the factory scene kept updating. It gradually improved, and in the end, I can say it became the polished HDRP scene I had in mind.

USD export at runtime

[15:18] The last thing I will mention is that we did not stop at URP and HDRP. I handle digital twin projects for other large enterprises as well, and what they most often say is, "Other teams or other departments in our company want to use this well-organized, simulation-ready data in other tools or on other platforms."

[15:44] So what we worked on was enabling USD export at runtime. Runtime export means that while running a simulation, you can change the layout to some extent, save it, and export that exact state in the USD format. That way, on another platform, textures, geometry, and everything else can be preserved exactly as-is and used right away.

Feeding the twin into world models

[16:18] Once you have a digital twin like this, it is not just about hooking things up and watching a dashboard. I also want to talk about today's hottest topic: world models.

With world models such as FLUX, Qwen, or NVIDIA Cosmos 3, you can take the screen from the digital twin you built in Unity, input that screen or image, and from there show things like the factory aging, steam coming out, sim-to-real scenarios, or even changes in the time of day. We were able to obtain a variety of datasets for these areas.

Project results in numbers

[16:55] To put our results in numbers: the project ran 14 weeks from kickoff. We organized 627 assets down to 218. We had 77 source code check-ins, and went from point clouds all the way to HDRP. Among the 39 datasets we had, we visualized 20 of them. Later, we carried out data augmentation based on more than 10 physics-based world models.

Side-by-side comparison and the manufacturing domain gap

[17:27] We made a final comparison video. Here you can see the point cloud first, then URP in the middle, and then HDRP. After that we ran diffusion, world-model simulation, and data augmentation.

There is one thing I would like to point out in this process. Models trained on general data have little grasp of manufacturing-domain data. So we spent a good deal of time studying how to close that gap for data in the manufacturing domain. This part will be explained next by Dr. Hongin Won.

I would like to thank Dr. Hongin Won, Researcher Youngseok Han, Researcher Jaehoon Hwang, and many others who worked on this project with us.

KITECH: turning a digital twin into an environment where AI can learn

[18:47] Dr. Hongin Won: I am Hongin Won from the Korea Institute of Industrial Technology.

Earlier, Manager Woojin Park explained how to build and expand a factory digital twin in Unity, specifically for a casting plant. As the next step, I will be talking about how to turn that digital twin into an environment where AI can learn and be tested.

The title of today's presentation is a Unity digital twin-based synthetic data pipeline for manufacturing AI. Since the title mentions a synthetic data pipeline, you may be thinking, "So how exactly do they synthesize the data?" But before that, the most important question is, "Has the digital twin really been built to match the real system?" I would like to focus a bit more on how we verify exactly that. We called the process of building a digital twin model to match reality "grounding," and I will explain it from that perspective.

Team and research direction

[19:49] I am Hongin Won, and I lead the Manufacturing AI Collaboration Team at the Manufacturing AI Research Center at KITECH. My areas of expertise are manufacturing AI, digital twins, and industrial data infrastructure. In particular, my area of interest in digital twins is virtualization, or how to move real-world problems into a virtual environment; generation, how to create the data needed within that environment; and validation, how to bring the results back into real-world verification.

[20:18] The main researchers who took part in this project are Researcher Jaehoon Hwang and Researcher Seungyeop Ha from our center. Researcher Jaehoon Hwang was responsible for measuring and correcting the remaining differences after moving sensors from the real environment into the twin, while Researcher Seungyeop Ha handled human-robot simulation and motion, as well as expanding the data across a wide range of conditions.

[20:48] Our research direction for manufacturing digital twins comes down to a single line: digital twins for AI, AI for digital twins. It means creating environments where AI can learn and be tested, and using AI to reconstruct and update digital twins. We are also researching advanced digital twins based on multi-agent systems and LLMs.

[21:09] Here is a short video on models our center has built. The video introduces models that transfer EV assembly lines, urban logistics, foundries, and more into twins and connect them as a single supply chain. We use the Unity simulator a lot, but on the back end, we also model scenarios with lighter-weight simulations, and the outputs from those are then brought into a Unity integration pipeline. That is a major direction of our research.

[21:48] The video also introduces the simulation and validation technologies developed there, including sensor fusion and studies where LLM-based agents plan robot paths. It also includes an example of planning robot paths using Unity MCP features.

Human-robot collaboration and why HRC data is scarce

[22:09] Today's topic is building environments where AI can learn and be tested. The scenario covered in this presentation is a manufacturing site where humans and robots collaborate. An environment where humans and robots work together in the same space is called human-robot collaboration, or HRC.

[22:25] As for the flow of the presentation, I will first talk about why this kind of HRC data is so scarce in the real world, and why we still need to build it all the way into the digital twin model. Then I will introduce the approach we took to overcome this, as well as the data synthesis methods we developed. And when building a digital twin model, many parts do not quite match the real world. In other words, a real-to-sim gap occurs, and I will also explain how we calibrated and resolved that gap.

[23:02] Let's start with the data problem. As you all know, AI learns from data. But some fields have plenty of data available, and others do not. For example, in autonomous driving, we can easily obtain millions of kilometers of driving records, and for general vision data, we can pull images and video from the web or from everyday life. Language models can also use data from across the entire internet for training.

But for manufacturing sites, especially for humans and robots working together, we need data where workers approach close to robots, parts of the body are occluded, or people move in and out of safety zones. That kind of data is extremely hard to find in public datasets.

Four reasons for data scarcity

[23:54] We identified four reasons for this data scarcity at manufacturing sites.

First is safety. The moment a person enters a robot's hazard zone is not something you can stage over and over just to collect data.

Second is cost. Setting up an actual line, installing sensors, and filming while changing conditions takes a great deal of time and money.

Third is the part we struggled with the most: labeling. For AI training, annotations and labels are needed to create the ground truth. But aligning the 3D positions of humans and robots, distances, joint information, backgrounds, objects, and pixel-level regions across multiple sensors at the same instant amounted to sheer manual labor.

[24:36] Last is rarity. Scenario rarity, to be precise. It is extremely difficult to obtain data on collisions between people and robots in manufacturing sites. Situations right before a collision are called long-tail events. Those situations almost never happen in reality, and they are extremely difficult for us to create artificially. And at a well-run site, such data should occur less often, not more.

Proximity data: what we actually need to train

[25:16] What we wanted to train was not simply whether a person is present or not, but how far the person is from the robot, in which direction, and what posture they are approaching in. This kind of information is what we call proximity data. We need to understand this relationship in order for humans and robots to collaborate and judge safe zones, and in the case of robots, to keep watch on how close a person is, or to create scenarios where they can slow down and stop.

[25:43] To put it simply, the situation was this. There was far too little data. So if we cannot collect it, generate it. That is what we set out to do. But to generate data, the model doing the generating needs solid data of its own. So first, we took values measured in the real world as our reference, and built a Unity digital twin synthetic data pipeline.

Why we chose Unity: four technologies in one runtime

[26:09] We felt it was relatively easy to connect four technologies within a single runtime, so we used Unity.

First is physical simulation, where robots, people, and objects interact in a physically valid way. Second is HDRP rendering, which matches lighting, materials, and so on to reality in order to reduce the domain gap. Third is sensor simulation, which reproduces several types of sensors virtually. Fourth is applied technologies that generate data from the twin so annotation and labeling can be performed without people marking everything manually.

[26:48] If these four had stayed separate, we could not have produced data for AI training. By integrating them into the same execution environment and the same time axis, we were able to generate the data we needed.

The Industrial HRC-Bench dataset

[27:03] The process of obtaining that training data had to follow a professional, reliable methodology for building datasets. To do this, we brought in experts from the manufacturing domain, and we jointly designed real human-robot collaboration scenarios. We created the authoritative experimental environment at the Robot Testing and Certification Center of the Korea Testing Laboratory.

[27:33] As a result of these experiments, we built the Industrial HRC-Bench dataset. The HRC scenarios collected so far, at a level ready for public release, come in two types: palletizing, and inspection of production parts. The dataset consists of a total of 17 episodes. Of these, nine were run with humans and robots working together, and the remaining eight with the robot alone.

[28:02] The sensor system used here integrates RGB cameras, LiDAR, 360-degree video, and a motion capture system. These sensor modalities were all synchronized at 20 Hz, which gave us more than 100,000 raw frames of data. When calculated in pure time, this corresponds to more than 80 minutes.

What mattered to us here was not only the scale of the data but its structure. All of this data shared the same time axis for observations and labels, which made accurate comparison and meaningful data generation possible. We plan to release the Industrial HRC-Bench dataset soon via Hugging Face or an external repository.

The pipeline in three words: ground, calibrate, generate

[28:53] I think the overall flow can be summarized in three words: ground, calibrate, generate.

The first is not simply creating a twin, but anchoring values measured in the real world to it. That is grounding, which aligns the twin with reality. Then there is calibration, which reduces the real-to-sim gap. And from there, the data generation phase.

Grounding: environment, sensors, robots, and labels

[29:35] Jaehoon Hwang: I am Jaehoon Hwang, and I was in charge of the real-to-sim implementation. In the grounding stage, there were four things we brought over from reality: the environment, sensors, robots, and labels.

Environment

[29:49] We measured the spatial dimensions and major equipment layout of the KTL testbed, and based on those values, matched the equipment and work areas one-to-one inside Unity. On top of that, we applied HDRP to match the materials and lighting to the real environment. For the background, using 360-degree panoramic images captured on-site, we built a photorealistic 3D Gaussian splatting scene to reduce the real-to-sim gap.

Our ultimate goal was to create a reference cell where the structure and occlusions the camera sees, how objects sit relative to each other, and how light and materials affect what is observed could be compared side by side with the real world. In the image on the screen, the left side is the real scene, and the right side is the digital twin from the same viewpoint.

Sensor rig

[30:34] For the experiment, we used RGB and depth cameras, LiDAR, a 360-degree camera, motion capture, and data from the two robots' states. Within the virtual environment we built, we did not simplify this into a single camera. After checking where each sensor was installed in the real rig and what it observed, we built virtual sensors with the same structure.

[30:59] Inside the Unity runtime, we built custom sensor components so that all observations would share the same simulation clock. We also connected the robot states and human motions so they would be synchronized at the same moment. The reason this synchronization is important is that proximity cannot be summarized with just a single image. At the same moment, video, depth, robot joints, and human pose all need to exist together in order to calculate distance and safety-zone labels consistently.

Robot

[31:27] We had one standard. It had to be motion that is physically valid, not motion that merely looks plausible. We brought in joint trajectories recorded in the actual testbed and configured them to replay frame by frame in their original time order. Using Unity's Articulation Body, we configured the robot's links and joints, degrees of freedom, and physical structure, and then ran the recorded joint states on top of that. Because inertia and contact are calculated together, we were able to handle the interaction between robot motion and surrounding objects within a single physical structure.

Labels

[32:02] From the same simulation state, four types of ground-truth information are generated together: 2D and 3D bounding boxes with the positions of people, robots, and parts; semantic and instance segmentation that separates objects at the pixel level; joint coordinates used for pose estimation; and depth ground truth used as a reference for proximity distance. The screen shows a scene with 3D bounding boxes applied.

These labels were not produced by someone marking up every frame by hand. They come directly from the Unity simulation state. That cuts both labeling cost and annotation error at the same time.

The real-to-sim gap in the camera

[32:52] Next, I will explain the real-to-sim gap we encountered while building the digital twin model, and how we addressed it. These are the residual gaps, the real-to-virtual differences that remain even after the transfer. Among them, we identified two factors that directly affect the proximity values and the ground-truth source.

The first one occurred in the camera. When constructing the virtual environment, we set the same sensor model and the same field of view for both the real and virtual setups. But the same object did not appear on the same pixels.

[33:20] If you look at the edge overlay on the right, you can see the boundaries of the same structure are slightly misaligned depending on the position. Even applying the actual lens data sheet to the Unity environment did not resolve it. That is because product specifications and standard lens models alone cannot explain the differences left by the installation angle and each lens.

In HRC, even a misalignment this small matters. If the boundary between a person and a robot shifts by just a few pixels, the correspondence between pixel labels generated in simulation and real observations also becomes unstable. So we decided to directly measure the residual of the exact camera installed in this testbed.

Measuring lens distortion instead of modeling it

[34:01] The method we settled on is simple. Do not model the lens directly. Measure it.

This process consists of three steps. First, using a 3D Gaussian view generated from multiple viewpoints, we obtained corresponding scenes in the real and virtual environments. Then we calculated the differences between those paired images at the pixel level and recorded them as a per-pixel distortion map. Finally, we applied that map to Unity's camera distortion shader so that the virtual camera is corrected during the image-rendering process.

[34:35] The key is this loop. We create corresponding scenes, measure the remaining difference, and feed that value back into the Unity runtime.

Here is the result. On the left is the observation from the real sensor, in the center is the corrected digital twin, and on the right is the edge overlay of the two images. Look at the boundary lines on the right and you can see a marked improvement over before. By applying a specific distortion map, we were able to confirm that the pixel alignment between the real and virtual images improved properly.

[35:09] This is not external post-processing but a component that runs when the virtual camera generates images, so the corrected observations and the ground truth can be produced within the same execution environment.

Fixing unstable human motion with IK

[35:16] The second problem appeared in human motion. The screen shows human motion recorded with motion capture, replayed with markers and a skeleton. This is a section where part of the body was occluded by the robot and the workbench during capture. Please watch how the joints of the foot start to shake. As occlusion grows, estimates of the hidden joints become unstable, so the foot slides on the floor and the joints move into physically impossible positions.

[35:47] This shaking directly distorts the distance between the human and robot, the proximity of each body part, and the safety-zone labels. So we applied physical constraints to the human motion as well. Those are the floor and the joints' range of motion.

This is the same scene as before. This time, there are just two things to watch for. Does the foot stay on the floor, and do the joints stay natural?

[36:14] First, with Foot IK, we reattached the foot to the measured real floor geometry. Then, using Humanoid IK, we constrained the occluded joints to move within valid joint limits. IK recalculates the positions of the joints in between, based on target positions for the hands or feet. This correction aims to reduce non-physical motion that disrupts proximity and safety labels.

Scenario playback: palletizing and parts inspection

[36:40] Dr. Hongin Won: I would like to thank Researcher Jaehoon Hwang for laying out the key technologies for reducing the real-to-sim gap, from pipeline construction to sensor and IK correction. I will briefly show you how the digital model we built behaves, and then wrap up our presentation.

[37:21] The two scenarios we built as digital models are based on the same HRC environment, but differ in their task characteristics. The first is palletizing, and the second is surface inspection. Both were designed as scenarios that could plausibly occur at a manufacturing site. We will release the specific details separately later in the abstract.

[37:44] This is the palletizing case. In the palletizing model, what we recorded from the real robot earlier through the Articulation Body is integrated into this digital twin model. As you just saw, two types of sensor data, robot state information, instance masks, and segmentation masks are all synchronized and played back together.

[38:10] The second is a scenario for parts inspection. The close-contact situation between the worker and the robot is also modeled, and it is very well synchronized with the simulation and robot data, so we can replay data in which all four elements are grounded. If you look closely, even when the person overlaps with a part, you can see that segmentation works very well.

[38:41] The value of this data pipeline is not limited to building a one-off dataset. It can also be expanded and reproduced across many other industrial sites.

Domain randomization with the Unity Perception Package

[39:15] What we showed earlier is less a one-time dataset and more of a generative data pipeline that can continuously expand data. Based on the Unity Perception Package, we built a domain randomization model and defined parameter ranges and sampling rules for lighting, materials, cameras, and so on, so that realistic variations are generated around this precisely aligned baseline.

System summary and closing

[39:39] I think we can summarize everything with this figure. This is the structure of the overall system. On the left are target objects, human motion, robot trajectories, and so on. They are input elements that can be swapped out at any time, and the center is the core.

[39:53] A ground model that sets the baseline using measurements from the actual site, a calibrate model that corrects errors by task, and on the right, a model that auto-extracts multimodal ground-truth data, the generate model. Those are the three we built.

We showed how we create a twin based on the actual testbed and calibrate the key task differences, all the way through to generating synchronized data.

[40:29] The data we showed today was HRC, but in fact, we presented it as an application to prove the design principles of our pipeline. In the coming era of AI factories, where robots and humanoids can coexist, we believe our pipeline can serve as the simulation and data layer.

Our thanks go to the KTL Robot Testing and Certification Center, which provided the testbed environment and ran the experiments with us, and to Unity Technologies, which was with us from the very start and backed us fully.