It’s fall and its likely your candy dish is overflowing with various bite-size treats. From chocolate bars to sour candies to gummy animals, Halloween is a time of sugar overload for many in the United States. Think about your favorite candy for a minute. If it were unwrapped, would it be easy for you to identify it by sight alone? The answer is probably “yes” because it’s your favorite and you’ve seen it many times. But let’s say you’re a fan of gummy candies. How easily could you differentiate between a Snickers and a Milky Way?
Determining the difference between candy bars may not be a matter of utmost importance or have serious consequences for most people. But for some, identification of food and the entirety of ingredients is very important. Therefore, and perhaps unsurprisingly, there are artificial intelligence (A.I.) systems being created with the sole purpose of differentiating between different types of food.
When you think of artificial intelligence (A.I.), it’s unlikely food identification is the first thing that comes to your mind. You may recognize the ways A.I. has permeated many aspects of daily life to not only complement your activities, but also increase efficiency. And although it’s easy to see how A.I. has successfully integrated into our lives in a variety of ways, using it for food identification is relatively new. With the complexity of the food we eat, how do you train an A.I. system to accurately recognize the food in front of you? The answer, though complex from a technical standpoint, is rather simple from a practical one. Groups of users and programmers have to spend time training the system to interpret a given stimulus. Because A.I. relies on information input to develop smart thinking, users are often tasked with helping to train the system. This is done by providing feedback and information around each stimulus and correcting any misinterpretations. Recently, doc.ai launched an A.I. training program focused on mood. Participants are rewarded for taking selfies of facial expressions relating to a different mood: neutral, happy, sad, calm, surprised and upset. Multiple days’ worth of photographs and data from multiple users trains the A.I. system to detect small differences and subtleties to better predict mood based on facial expression alone.
But what about your Halloween candy?
As a joint project between doc.ai and Passio, your help is needed to train A.I. systems to better recognize the food (including candy) you eat. Passio is focused on combining machine learning with real user input to create an advanced method for nutritional tracking. This goes far beyond candy and even standard, single-ingredient foods like tomatoes, apples and rice. This A.I. module can recognize the diversity and complexity of the food and drinks you consume each day.
Take a look at this video for how to participate:
Tracking food intake has been commonplace for various reasons. People who have medical conditions that require careful attention to dietary requirements often need to track their food so they can easily and quickly assess their nutrition. Dr. Chethan Sarabu, Director of Clinical Informatics at doc.ai explains, “On a general level [food tracking] could be used for understanding which diet may be the most beneficial for overall heart health; or for more specific cases, which diets may work best for individuals’ conditions ranging from diabetes to inflammatory bowel disease.”
Beyond its use in treating various diseases, food tracking is actually a widespread practice. The majority of Americans indicate they have previously or would consider food tracking and this includes those without a medical indication. Recent studies have shown the value of food tracking when it comes to things like weight loss, and food tracking is a common feature of high-performance athletic training. Regardless of the reason or indication, food tracking is done to assure the right balance of food is consumed to meet dietary and performance goals.
Why does food tracking require A.I.?
Simply put, food tracking can be time-consuming and error-prone. Current food tracking methods often rely on manual entry and estimation of quantity. When it comes to the need for accurate tracking of food and drink, human error can be a significant hurdle. “A.I. software for food identification can play a powerful role in gathering this information by making it much easier for individuals to collect,” says Dr. Sabaru. If with the wave of your phone’s camera, A.I. software can provide accuracy in identifying ingredients and quantities, food tracking becomes easier and less thought-provoking. This has huge implications for the millions of people living with diseases directly related to the food and drinks they consume.
There are significant challenges to training A.I. modules for food identification. According to Dr. Sarabu, “A.I. food detection relies on using trained databases and these databases are generally missing wide ethnic and cultural representations of food.” Additionally, in the case of food cooked with various and numerous ingredients, it can be difficult for the software to accurately identify each one. That is why your participation is so important to the success of these programs.
Participation is simple. Each participant will be asked to capture the meals and food and/or beverage that they will be eating. After the A.I. software attempts to categorize and label the food, the user will need to confirm or correct the software’s description. Only the images along with the food and drink descriptions will be shared with Passio. No additional data to identify the user will be shared.
Similar to the other modules developed by doc.ai, this AI training is one more tool to assess the variety of factors that play a role in health and wellness. As Dr. Sarabu says “doc.ai has been designed to bring together health information across multiple sources (or omics) to better learn how these factors integrate for someone's health.” Your participation is crucial to allow better interpretation of the data collected.
How can I participate?
Go to the doc.ai app and sign up for the A.I. Training for Food interested list. Participants will be invited to join the training on a rolling basis over the next month.
So grab your phone and your candy bucket and help doc.ai and Passio make food tracking easier and more accurate for everyone!