Applying intelligent automation to the problem of pruning vines during the dormant season.

By George Kantor, Ph.D.

The CMU team is using robotics and sensing to address many important problems in vineyard production, but the effort I am most excited about right now is our effort to apply intelligent automation to the problem of pruning vines during the dormant season.  This work is being led by postdoctoral project scientist Abhi Silwal, and it involves a top-to-bottom approach that is developing the necessary perception, manipulation, and mobility capabilities in order to automate the entire process. The manipulation will be performed by a commercial robotic arm (Universal Robotics UR5) equipped with an extra degree of freedom and a custom pruning tool. For moving the arm around in the field, we will re-purpose our “Cave Crawler” robot, which is capable of agile motion on rough and muddy terrain. Abhi and his team are working hard in the lab to pull all of these pieces together in time to do an end-to-end demonstration of the concept during the current dormant season, hopefully sometime in February 2018.

One of our recent breakthroughs has been on the perception front. Here, we use the same imaging system that has been developed for crop load measurements to take pictures of dormant vines in the field. We use a deep learning architecture called Generative Adverserial Networks (GANs) to analyze the resulting images to detect the vines and estimate pruning weight. GANs are a recent breakthrough in artificial intelligence for image analysis. In a GAN, two neural networks play a game against each other. The first network (the “generator”) tries to artificially mimic the way a human would label vines and buds in an image by trying to match the distribution of training images that have actually been labeled by a human. The second network (the “discriminator”) tries to guess if a given labeled image is artificial (i.e., is created by the generator network) or a human-labeled image from the training set. During training, both networks evolve to improve their performance, getting better and better the more times they go up against each other. In this way, the generator network can learn to mimic a human with a relatively small human-labeled training set. This is a big advantage because labeling images is a time intensive process, and getting enough labeled images is typically the bottleneck in other deep learning approaches.  This is an important advantage, and we have hopes that the successes developed for vine detection can be transferred to improve our other perception efforts on the Efficient Vineyard project such as detecting and measuring fruit on the vine.

Applying intelligent automation

Applying intelligent automation

Cave Crawler Applying intelligent automation

Cave Crawler Applying intelligent automation

Universal Robotics UR5 Manipulator Applying intelligent automation

Universal Robotics UR5 Manipulator Applying intelligent automation