Artificial intelligence chef cooks like a 5-year old. Even comes with made up words.

You may not know this. I’m a food blogger and I also work at a software company. So that means I enjoy learning about new technologies (mostly), spend too much time in front of the computer, and am contributing to the evil trade of building robots that replace humans like you and me. 

Background

Every year, I attend the Amazon Web Services (AWS) summit to learn what’s the latest ‘n greatest in cloud computing, machine learning (ML) / artificial intelligence (AI), and smart devices. 

This year, I watched a presentation on AWS DeepComposer where deep learning algorithms transformed the children’s song Twinkle Twinkle Little Star into a rock tune, a Bach-sounding theme, and a jazzy jingle. 

“Can AI do this for me with recipes?” I wondered.

After all, technocrats tell us robots are plotting to takeover most industries, including manufacturing, medicine (radiologists), and housekeeping (check out robot butlers).

How hard could it be for me to build an algorithm that would generate amazing, perfectly seasoned, and mouthwatering recipes? 

Images of lounging at the beach with an icy lemonade in one hand and a sleek published cookbook in the other materialized in my head. 

Things went downhill soon after that fleeting and blissful daydream. 

---

The model

Two weeks later, I was in deep, cursing at how hard Anaconda was to install. Why had I decided to take on this stupid project? 

Turn after turn, there was another obstacle blocking me from my beachy visions and pomegranate lemonade. 

The sliver of gratitude I felt was directed towards all the smart and generous people who have written up their machine learning projects, including data and code to quickly replicate their AI-generated recipes. 

This includes Oleksii Trekhleb, who wrote this comprehensive guide on how to use his character-based Recurrent Neural Network (RNN) model to generate new recipes. I followed his recommendations and used the Recipe Box dataset (about 125,000 recipes) and this smaller Kaggle dataset (about 20,000 recipes). 

I combined those two datasets with the Recipe1M+ dataset from a research team at MIT. Javier Marin was so kind to give me access to over a million recipes. His research team includes Amaia Salvador, Nicholas Hynes, Aritro Biswas, Yusuf Aytar, Ferda Ofli, Ingmar Weber, and Antonio Torralba from a partnership between Universitat Politecnica de Catalunya, Massachusetts Institute of Technology, and Qatar Computing Research Institute.

---

Despite having with such a huge dataset, this is one the best recipes the model could come up with: 

---

A couple of observations: 

• The recipe starts out reasonably. The title “Pie Pudding II” even copies the trend of recipe titles you often see on AllRecipes.com, such as “Banana Bread IV”. It also doesn’t sound disgusting. 
• The ingredients list makes sense. There’s a hefty dose of added sugar in this recipe (brown sugar, marshmallows, and confectioners’ sugar). But American dessert recipes are usually cloyingly sweet, so nothing out of the ordinary there.
• Once we get to the instructions, the first line seems on track. Although, we begin to suspect things are going to proceed sideways with the mention of “cinnamon candies”. What candies? 
• Soon, I’m seeing ingredients in the instructions like “pumpkin puree” and “baking powder” that were never mentioned. How many carrots? How much pineapple and liqueur? 

This recipe would produce a worse result than if a chimp taught you how to bake a pumpkin pie. 

The model appears to have spelling and grammatical challenges too. Not only did it struggle to make a coherent recipe, but it reminds us of endearing toddlers who make up real-enough sounding fake words (“leconing”?).

---

My majestic visions of retiring to the beach near my parents’ house in New Zealand vaporized. 

Alex kindly informed me that it’s possible to create the amazing and splendid recipes I was looking for. But I’d probably need a PhD in Machine Learning and to dedicate the duration of a PhD (minimum ~6 years) to solving the problem if I were serious about getting my model to successfully produce compelling and real recipes that can be cooked in a kitchen. 

Despite this setback, I learned a lot about AI and cooking through the lens of seeing artificial intelligence as a metaphor for all of us home cooks.

What AI taught me about cooking

Recipes are complicated: Recipes are their own language with rules, terminology, and a rhythm that must be learned.

It takes a while to learn the language of cooking and how to correctly read a recipe. As you know, I believe there are plenty of poorly written and untested recipes on the Web. It takes skills to write a good recipe and avoid assuming too much about what the home cook can achieve.

AI struggled with learning the language because it would first need to be trained on how the English language works. Then have a huge dataset to train it on how to read a recipe. 

Cooking requires a lot of complex skills: Recipe reading is 1 tiny part of the cooking process.

Reading a recipe and making sense of the directions doesn’t include all of the auxiliary tasks required to prepare a meal. Those additional skills include: forecasting (meal planning), grocery shopping, recognition of ingredients, access to good tools & equipment, knife skills, heat management, timing, and seasoning.

It’s no wonder beginners feel overwhelmed! 

The AI doesn’t have the context outside of the recipes to learn what is involved in cooking. It can’t use all 5 senses and pattern recognition from engaging in the real-world to generate a fabulous recipe.

It can recognize patterns in text but cooking is so much about an “art” rather than pure “science” so the environmental, historical, cultural, and sensory context is important.

Substitution is hard: Beginners who modify the steps or ingredients often end up with a bad result. 

I consider AI a beginner—like a 5-year-old child who is learning for the first time. It has never made a recipe before. It doesn’t understand what the ingredients are doing or the how the technique works. 

You can see how the model failed to generate a coherent recipe because the model doesn’t understand basic principles of cooking.

On top of that, the model is trying to create a new recipe and substitute ingredients and cooking techniques to spin up something new. There’s not enough information to allow it to produce something great. 

What my model taught me about AI

We’re probably still far from having AI that can easily replace creative jobs. Poetry, comedy, recipe and writing are difficult for a computer. There’s a lot of layers of meaning and context to teach a computer. 

Learning the English language via recipes isn’t an accurate way to learn English. 

There is a lot of data cleaning involved in data science. Data science is the fun, sexy side, but the data governance, data quality, and pipelines (this is my day job) are the foundations to doing any kind of analysis, let alone modeling. 

We might not be able to rely on the AI to generate an original recipe. But it could be helpful to analyze huge data sets to understand how ingredients are paired together. It can do heavy computational tasks like scanning 1 million recipes that a human could never complete in her lifetime. 

AI will need a lot more inputs (training data) to make a useful recipe. This exercise would have been much better with ingredients databases, tagging recipes with cuisine types, cooking techniques, and flavor profile data sets.

These extra inputs add significant cost and effort to the AI project. This helps explain why it’s expensive for companies to add AI to their business processes (and it’s unclear what the ROI is, especially for small-medium businesses…if they can access these technologies).

There’s likely enough training examples to use if we combine all the knowledge at humankind’s disposal. However, it’s a Herculean effort to bring professional chefs, kitchen equipment manufacturers, food bloggers, food historians, and companies in food-related industries, such as restaurant and hospitality to grocery supply chains, to share data with each other to build out a comprehensive model. After all, this data is usually intellectual property and a competitive advantage. 

AI + humans could be the best combination. AI could provide the starting point (for example, analyze ingredient pairings that humans haven’t considered) and combine it with human’s creativity to invent something original. AWS DeepComposer in the context of music is a great example of this. And this group of based out of MIT that embarked on a project to create a Human-AI collaboration to create original pizza recipes with a pizza chef in Boston. 

For now, I’ve still got my food blog and day job. And I’m grateful.

I’m excited about what AI can achieve in a year, five, or ten. Beyond recommendations based on purchase history and recipe recognition based on a food photo, I know there are talented engineers and data scientists working on exciting tools.

For example, this company, Passio, has a smartphone app that can tell you the nutritional breakdown of your food from a photo taken when your smartphone. There are also more and more recipe recommendations.

Maybe one day, they can synchronize with grocery stores to give me a perfectly coordinated meal plan with built-in food delivery. 

I learned a great deal from this project. It gave me a new eye for reading recipes and a distinct gratitude to the recipe testers and developers toiling away in test kitchens. And I’m looking forward to the next wave of innovations. 

READ NEXT: How to Build Your Cooking Confidence in the Kitchen