In an industry that has long relied on the eyes of trained experts for everything from prospecting and assessing ore quality to monitoring employee safety conditions, new technology in the form of computer vision is watching, learning and ultimately helping to chart the future for safer and more efficient mining operations.

Computer vision is a field of artificial intelligence (AI) that trains computers to interpret and analyze visual data to accurately identify and classify objects. Computer vision systems—powered by cameras, sensors and machine learning—are already being used in the manufacturing, retail, automotive and health care sectors to eliminate the need for manual visual checks and to drive efficiencies.

At mine sites, the technology is showing great promise for everything from improving mineral recovery and waste reduction to curbing safety concerns of employees in the field.

“Artificial intelligence is replacing human eyes,” said Dhruv Sharma, founder and CEO of Convolve AI. In fact, the Waterloo, Ontario-based AI and analytics consulting firm is currently in talks with mining companies in Canada and abroad that are interested in piloting computer vision systems for a variety of applications. “The technology has advanced to a point where it’s good, reliable and usable,” he added.

Similar to the human eye, computer vision enables machines to “see” and interpret images and videos. Whether the technology is embedded into cameras, drones or satellite images, it can be trained to track movements, recognize objects, detect patterns and even make decisions based on what is in its line of sight in real time.

By training on thousands or millions of labelled images and videos (such as trucks or geological features at a mine site), the AI model eventually learns to identify distinguishing features and make predictions about new, as yet unseen images. The computer vision system is trained by repeated exposure to visual examples that build familiarity and expertise.

Sharma cited the example of a mining operation equipped with 50 video cameras. “Today there may be one or two people assigned to watch the cameras, but AI could be monitoring all those cameras at once without missing things,” he said. Plus, computer vision could be used to quickly process video feeds from all those cameras and highlight potential safety issues with faulty equipment or operational inefficiencies in need of fixing.

As AI continues to evolve, future applications for computer vision show even greater promise with the integration of natural language processing, noted Sharma. This provides the potential for employees to ask questions of particular computer vision systems in plain language to get actionable results (e.g., “show me unsafe areas in the last scan,” or “what does the drone see?”).

Mining applications

The technology is gaining particular momentum now because the computing power has reached a point where it can effectively capture data in a dynamic environment like mining, according to Ravi Sahu, CEO of Strayos, a provider of an AI-powered platform to enhance mining operations that is currently being used by some 150 Canadian mines.

“What once demanded sprawling server rooms now thrives on a compact GPU [graphics processing unit] tethered to a computer vision camera,” he said. “In a year from now, you’ll have even more computing power and more use cases emerging in mines.”

In terms of present-day applications of computer vision, Sahu pointed to autonomous mining equipment as a key use case. “If the haul truck driver can be in an office far away operating a truck, it’s always better in terms of keeping employees safer,” he said.

Similarly, five trucks equipped with computer vision technology would be able to sense each other’s production loads and feed that data to other machines on site. “Now I have this dynamic data and can go and make production changes as needed,” said Sahu.

More autonomous vehicles and other mining equipment may also be a solution for ongoing labour shortages in the mining sector. “As machines go more autonomous, we can start attracting a new generation of talent that won’t be spending 80 per cent to 90 per cent of their time in the mine,” he said. Rather, they will be designing systems and steering operations through software, hundreds of kilometres away.

Computer vision is also expected to significantly improve ore sorting and grade control. That is because AI-enhanced cameras can perform real-time ore analysis and automatically identify mineral composition and particle size on conveyors and shovels to enhance sorting. The technology can also be used to separate ore from waste, which would cut down tailings production and water consumption, improving the overall environmental impact of mining companies, said Nermina Harambasic, founder of O-MOD, a capital project oversight advisor and AI for mining adoption consultant.

“Over time, these new tools will lead to changes to ore processing facilities as new solutions will remove existing bottlenecks and transform current configurations,” she said. “For example, [computer] vision can be used at the front end to monitor ore fragmentation; separate ore and waste; send that information to the processing plant; and, as a minimum, optimize the existing process or, ideally, simplify configuration of the downstream process.”

Obstacles to adoption

Yet even with all its potential for improving operations and safety in mining, there are a number of factors preventing mainstream adoption of the technology just yet, said Peyman Sindareh, senior data scientist at MineSense Technologies in Vancouver, which provides companies with digital mining solutions. He highlighted limited infrastructure in remote areas, harsh environmental conditions and difficulty integrating with legacy systems as just some of the reasons why computer vision is not more broadly implemented in the mining industry.

“Environmental factors such as dust and poor lighting can lead to false detections, and AI models often require retraining to adapt to new sites and perform site-specific tasks,” he said. “Additionally, customizing and maintaining these systems can be costly.”

Developing effective computer vision models also requires large volumes of field data where security is a major concern, so the data needs to be reviewed by both computer vision specialists and mining domain experts. The labelling process itself is also costly and time-consuming, often requiring expert annotation, he said. Training accurate models then demands advanced expertise in computer vision and computational resources.

“Even after successful development, there’s a critical gap in training operational staff to understand, trust and effectively use these models to drive value and profitability,” said Sindareh. Those mining companies wanting to take full advantage of the potential benefits of computer vision should be investing in hardware, training, transparent data policies and careful change management, he recommended.

In spite of these hurdles, the long-term gains with computer vision will be worth the growing pains, said Sindareh. Compared to traditional models—such as visual inspection by operators, mechanical sensors with no image data, or rule-based systems that use fixed thresholds without adaptive learning—he believes computer vision solutions can be more robust, reliable and ultimately less expensive. For example, he said particle size distribution analysis using computer vision may offer a faster and more cost-effective alternative to conventional sieving methods.

The future

As computer vision becomes more advanced and technologies become better integrated, mining companies that adopt it can expect their productivity gains to only get better, said Yael Valdez, an optimization specialist for digital solutions at Hatch. For example, he said, a computer vision model that is actively tracking compliance or performance and is integrated with an optimization system in the background can harmonize to provide recommendations on how to alleviate bottlenecks at sites as soon as deficiencies are detected.

Similarly, an automated control and response process using computer vision could be used to improve safety in the field. If a person is detected within five metres of a moving haul truck, for example, the system could automatically trigger the truck to turn off, preventing a potentially fatal injury. “I think interconnecting different types of advanced analytics and even basic applications is where these computer vision models will really shine,” he said.

Beyond that, Harambasic pointed to advancements in humanoid robots equipped with computer vision as another big breakthrough to come in the mining sector. “Car manufacturers are already using humanoid robots in their pilot programs,” she said, noting that robots should ideally be used at the positions where people are exposed to harsh conditions and repetitive tasks or to assess potential danger before humans enter an area. “I think [these robots] will be a game-changer in many industries, including mining,” she said.

As with any new technology being introduced into a workplace, pushback from employees is inevitable, said Valdez, especially when adopting technologies that track or change how work is monitored. But he suggested that having clear communication around why and how it will be used, taking an operator-focused approach to change management, and ensuring that the organization’s employees and management are on the same page, would allow the deployment of computer vision to be a big step forward.

He noted that many mining companies are already housing comprehensive amounts of data from cameras that is not being used to provide analytics on operational bottlenecks or safety issues. “Right now, images and video are a form of unstructured data that is being highly underutilized by mining companies,” he said. “Those not leveraging or planning to leverage this type of data will be left behind.”