PROJECT STUDY
Rock & Coal Field Scanner
Screen in the field and let the lab decide.
An offline-first field app for Indonesian coal and mineral sites that turns a sample photo, field measurements and handheld XRF readings into an indicative screen with stated uncertainty.
Explore the build
At a glance
- Industry
- Mining
- Capabilities
- 6
- Technologies
- 12
- Year
- 2026
The challenge
Field geologists and quality-control staff at Indonesian coal and mineral sites wait days for laboratory assays, yet they decide on site which samples to bag, which to send for assay and where to look next.
They need a first read in the field: what rock this is, the likely coal rank and calorific range, and whether a sample shows signs of gold, copper or nickel worth an assay.
A phone photo cannot measure calorific value or gold grade, so a useful screen has to say how much it knows. It has to combine the photo with field measurements and handheld XRF readings, state its uncertainty, keep each sample's chain of custody to the lab, and learn from the lab results that come back.
What we built
Rock & Coal Field Scanner pairs an offline-first Expo app for the field with a Python screening service, and every value it returns carries its basis, its unit and its uncertainty.
Offline-first capture
A sample photo from the camera or gallery, the sample ID scanned from a QR or barcode label, GPS with its reported accuracy, optional proximate measurements and a handheld XRF export are saved to a queue on the phone, which syncs when coverage returns
Rock type as a prediction set
A texture classifier covers coal of low, medium and high rank appearance, sandstone, claystone, limestone, quartz vein and laterite, and returns every class that passes its conformal threshold at a target coverage, so an ambiguous photo yields two classes rather than one overconfident label
Coal quality with intervals
Calorific value comes from a regression calibrated on the site's own field-to-lab pairs and is reported with a prediction interval as GAR, GAD and NAR in kcal/kg (GAR also in MJ/kg), together with the candidate ASTM D388 ranks across that interval
Mineral flags from handheld XRF
Imported readings keep their detection limits and errors, and gold, copper, nickel, iron and sulfur are flagged against site-configurable thresholds as indicative anomalies. A gold flag is a request for fire assay, never a grade
Chain of custody
Each sample is recorded as captured, synced, sent to the lab under a batch number and assayed, with every step kept against its sample ID
Lab reconciliation
Lab results imported by sample ID are compared with the estimate the field team saw, each import refits the site calibration as a new version, and the lab view flags drift
Offline-first capture
A sample photo from the camera or gallery, the sample ID scanned from a QR or barcode label, GPS with its reported accuracy, optional proximate measurements and a handheld XRF export are saved to a queue on the phone, which syncs when coverage returns
Rock type as a prediction set
A texture classifier covers coal of low, medium and high rank appearance, sandstone, claystone, limestone, quartz vein and laterite, and returns every class that passes its conformal threshold at a target coverage, so an ambiguous photo yields two classes rather than one overconfident label
Coal quality with intervals
Calorific value comes from a regression calibrated on the site's own field-to-lab pairs and is reported with a prediction interval as GAR, GAD and NAR in kcal/kg (GAR also in MJ/kg), together with the candidate ASTM D388 ranks across that interval
Mineral flags from handheld XRF
Imported readings keep their detection limits and errors, and gold, copper, nickel, iron and sulfur are flagged against site-configurable thresholds as indicative anomalies. A gold flag is a request for fire assay, never a grade
Chain of custody
Each sample is recorded as captured, synced, sent to the lab under a batch number and assayed, with every step kept against its sample ID
Lab reconciliation
Lab results imported by sample ID are compared with the estimate the field team saw, each import refits the site calibration as a new version, and the lab view flags drift
How it works
Capture without signal
Each capture is written to a local queue under a client-generated capture ID, in SQLite through expo-sqlite on the phone. When coverage returns, the queue syncs in capture order and backs off on transient errors. The service treats the capture ID as the key of the sync, so a retry returns the stored result, while a different capture that reuses a sample ID is refused because two bags must never share a label.
Screen on the server
When a sample syncs, the FastAPI service decodes the photo, crops it to a centred square and extracts colour, grey-level co-occurrence, local binary pattern and gradient features. ONNX Runtime scores the eight classes, and the conformal thresholds turn the scores into a prediction set. A feature guard flags photos that look unlike any rock surface the model was trained on, and photo hints call out low resolution and washed-out highlights.
Estimate coal quality
The site calibration predicts GAR from dry, ash-free volatile matter, total moisture and as-received ash, with a Student-t prediction interval that widens away from the data the site has measured. Other bases convert through the ISO 1170 relationships, NAR follows the ISO 1928 calculation and rank follows ASTM D388 with the Parr formulas. Dulong's formula serves as a cross-check when an ultimate analysis comes with the sample.
Reconcile with the lab
Samples go to the lab under a batch number, and the results come back as a CSV matched by sample ID. Each import closes the chain of custody and refits the calibration as a new version, so history is never rewritten. The lab view plots estimate minus assay for every sample, checks rock prediction sets against the logged lithology and reports drift when recent lab values fall inside their intervals less often than the stated level allows.
Train, calibrate, bundle
The ml package renders procedural rock-surface textures for every class, with wet, dusty and shadowed variants and phone-camera effects, so the whole path from photo to prediction set can be trained, calibrated and tested end to end. It fits a scikit-learn classifier on the texture features, exports it to ONNX with skl2onnx and computes the conformal thresholds on a held-out split.
Engineering insights
The scanner never returns a bare answer: rock type is a set, calorific value is an interval, and rank lists every ASTM D388 group the interval allows.
Stack by layer
- Frontend
- Expo
- React Native
- TypeScript
- Expo Router
- expo-camera
- expo-location
- expo-image-picker
- expo-document-picker
- Backend
- Python
- FastAPI
- Pydantic
- Uvicorn
- Data
- SQLite
- expo-sqlite
- AI
- ONNX Runtime
- scikit-learn
- skl2onnx
- NumPy
- SciPy
- Pillow
- Infrastructure
- uv
- pytest
- Hypothesis
- Vitest
- Biome
- Ruff
- mypy
A prediction set per rock class
Conformal thresholds are computed per class, so a rare, decisive class cannot hide behind a good average, and a class with too few calibration photos for a finite threshold is always included rather than dropped. A set of two classes tells the geologist that the photo alone cannot decide
Thresholds locked to their model
The ONNX graph, its thresholds and the feature guard travel in one bundle, each pinned by SHA-256 in a model card, and the service refuses to start on a mismatched graph, feature recipe or class order. Conformal thresholds are valid only for the exact model they were computed on
Calibrated on field inputs
The calorific regression is fitted on pairs of field measurements and lab results for the same sample, so its interval absorbs the error of the field kit and not just the lab's. Each refit is checked with whole pit blocks or drill holes held out, because neighbouring samples are correlated and a leave-one-out check would flatter the model
Every value carries its basis
Values are stored with their basis and converted only through the dry basis, so a GAD lab value is compared with a GAR specification after conversion. NAR is never scaled from GAR, because it needs hydrogen: without an ultimate analysis it becomes a range drawn from the site's own hydrogen history, and rank uses the site's equilibrium moisture when the lab has measured it
Censored XRF stays censored
A reading below its detection limit stays censored, never turned into zero or into the limit, and a flag counts the reported error, so only a reading whose lower bound clears the threshold is indicative. Arsenic is marked uncertain where lead is higher, because their main X-ray lines overlap
Design system
Field kit treats the app as a field instrument rather than an office app: high contrast for direct sunlight, controls sized for gloved hands and one primary action per screen, so a geologist at a pit face or in a core shed can record a sample without hunting for controls.
Colour palette
- Basalt
#1c1917Text and icons, including every label on lime and gold fills - Chalk
#fafaf9Screen background, and banner text on coal - Sandstone
#d6c7a1Input and chip fills, range tracks and dividers under basalt text - Coal
#262626The permanent screening banner and photo backgrounds - Ore gold
#b8860bMineral flags only, as a fill behind basalt text, never as text on chalk - Signal lime
#65a30dThe single primary action on each screen, with basalt text and a basalt outline - Alert red
#dc2626Errors, failed checks and lab values outside their interval, always with a label
Colour
Basalt text on chalk, with colour reserved for meaning: signal lime only for the primary action, ore gold only for mineral flags and alert red only for problems, each paired with a text label. A coal banner reading "Indicative screening, not an assay" stays at the top of every screen
Typography
Atkinson Hyperlegible, the Braille Institute's typeface for low-vision legibility, sets all text at large sizes with bold values. Its distinct letterforms keep sample IDs that mix letters and digits unambiguous, and every value prints with its basis and unit
Components
A small in-house set on React Native primitives: a full-width lime primary button with a basalt outline, pinned in a footer below the scrolling content, segmented choices instead of dropdowns, numeric fields that show their unit, range bars for intervals, and chips for sync and sample status
Accessibility
Touch targets are at least 48 dp, and 56 dp for the primary action. Tests check every text pairing against the WCAG 4.5:1 contrast minimum, and because lime alone barely separates from chalk, the primary button always carries a basalt outline
Where this fits
Mining, quarrying and exploration teams that wait on assays can use the same pattern: a fast first read in the field, clear about its uncertainty and tied to the lab results that confirm it. Explore our mobile apps, AI solutions and computer vision services, or talk to us about your field workflow.
KEEP EXPLORING
Another problem. A different approach.
Spatial Digital Twin
An editable 3D planning workspace for comparing room layouts, checking clearances, and simulating queues before a space is rearranged.
Store & Workforce Vision
A privacy-aware store operations platform for occupancy, queue and zone events, with configurable alerts, manager review and aggregate reporting.
Let’s connect the dots
Let's put these ideas to work.
Share a process, product, or problem you want to improve. We will work through the possibilities together.