Fitterfly, with its latest partnership with Google Cloud on Fitterfly Klik, an AI-enabled feature that helps people with diabetes and other non-communicable diseases track and analyze their meals, aims to extend its functionality to help subscribers make healthier food choices while purchasing groceries.

“Currently, Fitterfly Klik simplifies meal logging by analyzing food on the plate.  However, we want to help subscribers even earlier—specifically when purchasing groceries. While walking through a supermarket aisle with Klik on your phone, you can point at the shelves. The app will provide real-time guidance on the healthiest options. This proactive feature will help users make informed choices during purchase, and promote healthy eating habits,” said Ammar Jagirdar, head of Fitterfly X-Labs.

The company’s latest feature, Fitterfly Klik, uses Gemini 1.5 Flash on Google Cloud’s Vertex AI platform to make meal tracking efficient, he said.

Gemini 1.5 Flash helps Fitterfly Klik analyze meals through object detection and image classification techniques. The tool has been trained on Fitterfly’s dataset of food images, enabling it to distinguish between a variety of foods with accuracy, claims the company. “This enables it to accurately differentiate between similar-looking items and correctly identify even the most obscure or complex dishes,” he added.

Fitterfly Klik uses AI computer vision paired with the Fitterfly Nutrition Database, comprising over 37,000 Indian foods, to recognize, analyze, and break down food. Within the Fitterfly app, users can access Klik within the meal-tracking food diary by clicking on the camera icon or through a chat with JEDi, Fitterfly’s AI Coach. The app will provide feedback on that food item, its portion size and give detailed nutritional breakdowns like calorie count, macronutrient and micronutrient distribution.

“Gemini 1.5 Flash, with its advanced multimodal capabilities, has played a significant role in enhancing the functionality and performance of Fitterfly Klik, particularly in food identification. Specifically, the enhanced image understanding of Gemini 1.5 Flash has been instrumental in building the food camera part of Klik. This advanced capability is superior to Gemini 1.0, allowing the system to accurately recognize, analyze, and categorize a wide variety of foods,” said Subram Natarajan, director of customer engineering at Google Cloud.

Jagirdar stated that Google Cloud’s AI expertise and machine learning (ML) capabilities have played a significant role in optimizing the performance and accuracy of Fitterfly. As a recipient of the Scale program grant, the company benefited from the early-stage support provided by Google Cloud, crucial in the product development phase.

“The Customer Engineering team helped us identify the most effective ways to apply AI and ML techniques to address specific needs, leading to improved performance and efficiency,” said the head of Fitterfly X-Labs.

Related Topics