An interactive excavator demo made in Qt Quick 3D in less than two hours 

We are experts in Qt; I know it sounds trivial, but let me tell you how we made an interactive excavator demo in Qt Quick 3D for our client in less than two hours. We share the demo in this article so you can see for yourself.

3D excavator demo for a geomatics company 

As part of our partnership with The Qt Company, we received a contact for a firm that develops solutions for the geospatial, construction, and agriculture industries. One of the client’s products is hardware and software for auto-steering and automatic control of agricultural or heavy equipment, such as tractors or excavators. 

The products aim to lower the entry threshold for machine operation and increase work efficiency by making the equipment use more convenient. 

The need to modernise the existing system 

The client already had an automatic remote-control system consisting of mobile devices such as a GNSS sensor, camera, or tablet in the operator’s cab for setting up machine parameters. 

The weakest link in this advanced product was the HMI software for the tablet – not transparent enough, presenting the machine only in 2D view. The client wanted to modernise the UX and UI layer of the application to make it more attractive for the global market. They decided to leverage the Qt technology to create a better interface. 

Let’s prove the concept of digging with Qt Quick 3D 

To test whether the technology would meet the client’s requirements, we’ve decided on a proof of concept – a sample view of the excavator control panel. The HMI application demo was intended to showcase our skills in this technology and present a vision for upgrading the existing user interface. 

In the PoC development process, we took the following requirements into account: 

• introduction of 3D views of the machine, 
• option to move excavator components remotely concerning its parameters, 
• implementation of three views: top, side and free float with screen zoom and rotation, 
• the development of different levels of resolution for the low/mid/high model in order to easily switch between devices, 
• no fixed values and flexibility of the application, knowing that the software would be used for other vehicles and tools. 

An interactive demo implemented in less than two hours 

The main HMI view was inspired by the client’s existing product. Without introducing a revolution in the functionalities, we redesigned the view to increase the user experience and make it easier to navigate. 

The UI project, designed in Figma, was imported to Qt Design Studio alongside a bought 3D excavator model. Using the WYSIWYG editor to implement simple functionalities and QML code for more complex components, we made all the elements come together as an interactive demo. Even though our Qt experts say they’re more adept at 2D, they hardly had any issues while working in the 3D environment. That’s mostly because all the changes introduced in the code are visible in real-time in the demo preview. 

Thanks to the partnership between Spyrosoft and The Qt Company, we have huge support from the technology provider, as well as broad access to their products, including beta and premium ones. In case of any problems or need for a consultation, we can count on specialists from the Qt side. 

For the client to interact with the demo via an internet browser without needing to install data locally, we made the demo available in WebAssembly format. 

You can try the demo here.

3D excavator demo with future development in mind 

As this article is about proof of concept, we could end it here. Nevertheless, I’d like to cover a few more topics about the project’s future development. 

First, safety features. While developing an interactive HMI application to control a machine, we implement components responsible for rendering critical safety interface elements. Those are speedometer, tachometer or, in the case of an excavator, information about a failure in the machine itself or a threat to its operation. If a failure or error occurs in the graphical interface, the safe renderer will provide the basic interface with the data necessary to drive the machine safely. 

Second, the digital twin. A digital twin is a virtual copy of the physical equipment that simulates its operation. It takes into account the parameters of the machine but also external factors such as slope, loading weight or weather conditions. The digital twin allows us to visualise the actual use cases of the machine operation under different conditions without testing “in production” from the operator’s cab. 

Third, cloud. Combining the project with the computing cloud allows communication between the product’s elements – the tablet with the parameters entered by the operator, the satellite tracking the machine’s location, and the device supporting its movement. Involving the cloud makes the software more efficient; it allows it to upscale when serving more customers and downscale when the load is reduced, lowering the cost by fitting it to the actual level of cloud resources consumption. 

Looking for experienced HMI development specialists? 

Has this project grabbed your attention? If you’re thinking about creating a remote device control system or improving existing HMI software, get in contact with us. Find out more about HMI development services provided by Spyrosoft, here.