# Synthetic Data: Training Vision Models When Real Data Is Scarce > How synthetic data for computer vision fills gaps when real images are scarce: rendering, compositing and generative methods, and how to prove they work. - URL: https://weaveward.com/blog/synthetic-data-computer-vision - Published: 2026-07-16 - Updated: 2026-07-16 - Category: Computer Vision - Tags: synthetic data, computer vision, training data, domain randomization, model evaluation - Author: Weaveward Team You can train a useful vision model with synthetic images when real ones are scarce, but you should not trust it until it has been tested on real ones. The pattern that works is simple: generate images for what you cannot easily photograph, mix them with the real images you have, and judge the model only on real photos from your own cameras that it has never seen. That last rule matters more than any rendering technique. Rendered pixels differ from camera pixels in lighting, noise, blur, compression and texture, so a model can score well on synthetic test images and still miss the real defect on your line. This guide explains when synthetic data is worth the effort, the four main ways to produce it, how to validate the result and how to run a first pilot without burning months on graphics work. ## When real data is the bottleneck Most vision projects do not stall because of the model architecture. They stall because the team cannot collect enough labeled examples of the thing that matters. Synthetic data helps in five recurring situations. ### Rare events A well-run production line produces few defects, which is good for business and bad for training data. If you are planning [visual quality inspection on a production line](/blog/computer-vision-quality-control), expect defect images to be the scarcest input. Safety events such as falls, forklift near-misses or smoke are similar, and you cannot stage them safely. ### Things that do not exist yet A retailer may want its shelf cameras to recognize a product before it reaches stores, or a manufacturer may be changing its packaging next quarter. Rendering the new design from its packaging artwork lets the model learn it before the first real photo exists, which matters for use cases like [shelf monitoring for out-of-stock items](/blog/shelf-monitoring-out-of-stock-detection). ### Conditions you rarely capture Night shifts, heavy tropical rain, glare from a loading bay door and haze all change how objects look. Real datasets tend to over-represent ordinary daytime scenes, because that is when people collect them. ### Privacy limits Biometric data, including face data, counts as specific personal data under Indonesia's Personal Data Protection Law (UU PDP, Law No. 27 of 2022), which raises the bar for collecting and storing it for training. Synthetic people can reduce how much real biometric data you gather during development. A deployed system that identifies real people still needs its own legal basis and risk assessment. ### Expensive labels Drawing pixel-accurate masks around every scratch, bolt or fruit bunch by hand is slow. Synthetic images come with perfect labels, because the generator knows exactly where every object is. Synthetic data is the wrong first move when you could collect real data in a few weeks by mounting a camera and labeling carefully, or when the real problem is inconsistent labels. Fix those first. Our explainer on [object detection from model to deployment](/blog/object-detection-explained) covers the fundamentals. ## Four ways to create synthetic training images Start with classic augmentation: flips, crops, rotation, color shifts, blur and noise applied to your real images. It costs almost nothing and is standard in most training pipelines. Synthetic data begins where augmentation runs out, when you need new content rather than variations of the same pixels. | Method | How it works | Best for | Main risk | Effort | |---|---|---|---|---| | 3D rendering with domain randomization | Build 3D models, then render many variations with random lighting, textures, backgrounds and camera positions | Products, packaging and parts with known shapes; new SKUs | Images look too clean; 3D modeling takes skill | High upfront, low per image | | Cut-and-paste compositing | Cut real objects or defects from a few photos and paste them onto varied real backgrounds | Small objects, defects, rare items in busy scenes | Wrong edges, shadows or scale teach the model shortcuts | Low to medium | | Generative image models | Diffusion models create or edit images from text or reference images, such as adding rain or inpainting a scratch | Weather, lighting and appearance variation | Labels may not match what was generated; subtle artifacts; license terms | Medium | | Scene simulation | Simulate whole environments with motion and physics in a game engine or robotics simulator | Traffic, warehouses, robots, safety scenarios | Heavy setup; realistic behavior is hard to model | High | ### Domain randomization in practice Domain randomization makes synthetic images deliberately varied, even unrealistic, so the model learns the object instead of the rendering style. You randomize light direction and color, textures, backgrounds, camera distance and angle, and add distractor objects. If the variety is wide enough, the real world looks like one more variation. Tools range from open-source 3D software such as Blender to game engines such as Unity and Unreal Engine. ### Generative models need label checks Image generation models produce realistic variations quickly, but they do not always put things where you asked. If you inpaint a dent into a car door, check that the dent actually sits inside the box you label. Spot-check generated images by hand, record the model and settings used, and read the license terms before using the output commercially. ## Validating on real images: the rule that decides everything A synthetic data computer vision pipeline is only as good as its real-world test. Build that test before you generate a single image. 1. **Collect a real test set from the target cameras.** Use the same cameras, angles and conditions as production. Include as many real examples of the rare class as you can find, even if that is only a few dozen 2. **Never train on it.** Keep it away from training and from tuning decisions, or your scores will flatter you 3. **Compare three models.** Train one on real data only, one on synthetic data only and one on the mix. If the mix does not beat real-only on the real test set, the synthetic data is not helping, however good it looks 4. **Break results down.** Report precision and recall per class and per condition, such as day versus night or dry versus rain. An average hides exactly the cases synthetic data was meant to fix 5. **Run in shadow mode.** Let the model watch live video for one to two weeks without triggering alerts, then review what it flagged and what it missed Our guide on [how to evaluate AI accuracy before you deploy it](/blog/evaluate-ai-accuracy-before-deployment) goes deeper into metrics and acceptance thresholds. ### Mixing ratio and fine-tuning There is no universal ratio of synthetic to real images. A common pattern is to pretrain on a large synthetic set and fine-tune on the smaller real set, but treat the ratio as a setting you tune against the real test set. As real examples accumulate from production, the synthetic share usually shrinks. ### Watch for shortcut learning Models are good at finding easy signals. If every pasted defect has a slightly sharper edge than its surroundings, the model may learn to detect sharp edges instead of defects. A quick diagnostic is to train a simple classifier to tell synthetic images from real ones; if it succeeds easily, the gap is large and worth narrowing with better textures, noise and blur. ## Running a first pilot A pilot should answer one question: does synthetic data improve performance on a specific failure that matters? Keep it that narrow. 1. **Name the failure.** For example, "the model misses torn seals on the night shift" or "the model confuses local helmet types", a common gap in [PPE detection on work sites](/blog/ppe-detection-safety-monitoring). 2. **Build the real test set** for that failure, as described above 3. **Pick the cheapest method that covers the gap.** Try augmentation first, then compositing, then rendering or generation 4. **Generate a small first batch,** train, and evaluate on real data before scaling up 5. **Inspect the errors** and adjust randomization ranges, distractors and labeling rules 6. **Feed real hard examples back.** Every false alarm and miss from shadow mode becomes a labeled real example for the next round ### A hypothetical example Imagine a snack manufacturer in an industrial estate near Cikarang that wants to catch torn or misaligned seals on pouches. A week of footage yields only a handful of real defects. The team cuts out those defect regions, composites them onto many normal pouch images under varied lighting and blur, and trains a mixed model. The go or no-go decision rests on recall measured against real defects collected over the following weeks, not on the synthetic scores. ### What drives the cost - The number of object types and how hard they are to model; shiny, transparent and soft items take longer - Engineering time to build and tune the generator - Labeling the real test set, which you need regardless - Compute for rendering, generation and repeated training runs - How many iterations it takes to close the gap between synthetic and real results ### Privacy, licensing and documentation Keep a short data card for every synthetic set: how it was produced, which 3D assets and models were used, their licenses and which real images it was derived from. Real photos of people used as seeds or backgrounds remain personal data under UU PDP and need the same protection as any other footage. If the model will run on cameras on site, our guide to [edge AI for computer vision](/blog/edge-ai-computer-vision) covers the deployment side. ## How Weaveward can build this for you We would start from the failure, not the technique: review your current images, your cameras and the cases your model misses, then collect and label a real test set from your own site. The first delivery would be a generation pipeline aimed at that gap, usually compositing or rendering, a model trained on the mix, and a comparison report against a real-only baseline broken down by class and condition. Before any alert reaches operators, we would run the model in shadow mode on live cameras and fold the real misses back into training. Explore our [computer vision services](/services/computer-vision), or [talk to our team](/contact) about the images you have and the ones you are missing.