Building a Recipe App with AI image Parsing - Why Markdown is Currently Better than JSON for Image Parsing Output

Building a Recipe App with AI: Why Markdown is Better Than JSON for Parsing Output

The rise of AI in app development has brought about a new way to interact with and generate content. One of the biggest challenges when building an AI-powered recipe app is ensuring consistent output. Many developers default to JSON for structuring AI-generated responses, but a lesser-known yet powerful alternative is Markdown. This only applies to openAI parsers that don't support object schemas. Currently chatGPT-4o-mini can only output text.

The Problem with JSON and Image Parsing in AI

While JSON is a common choice for structured data, it presents significant challenges when working with AI-generated content and trying to prompt a JSON output when it really just wants to return a markdown-esque text. The JSON output ends up being very unreliable due to:

Strict Syntax Requirements – AI models may generate incomplete or invalid JSON, causing parsing errors. Getting a response that adds additional content outside the json block, things like "here is your JSON ".
Escaping Issues – Special characters and newlines can lead to malformed responses.
Unexpected Formatting – AI may not consistently generate properly nested JSON structures, leading to data loss or misinterpretation.

Example: AI-Generated Recipe in JSON vs. Markdown

JSON Output (Prone to Errors)

{
  "title": "Spaghetti Carbonara",
  "ingredients": ["200g spaghetti", "100g pancetta", "2 eggs", "50g parmesan", "Salt, pepper"],
  "instructions": "1. Boil pasta. 2. Cook pancetta. 3. Mix eggs and cheese. 4. Combine all."
}

If text is added before the brackets, or If any comma is misplaced or a quotation mark is missing, parsing fails. Trying to get all the subtleties out of the text output is a nightmare. Even adding rules like "do not add text outside json" will return text like "Here is your JSON ". AI is stubborn and when it wants to give you a text output it will, I struggled with this for many hours and finally decided to try to just parse the markdown text chatGPT seemed to naturally produce.

Markdown Output (More Robust)

~ ~ ~ Spaghetti Carbonara

## Ingredients

- 200g spaghetti
- 100g pancetta
- 2 eggs
- 50g parmesan
- Salt, pepper

## Instructions

1. Boil pasta.
2. Cook pancetta.
3. Mix eggs and cheese.
4. Combine all.

Parsing Markdown in Your Recipe App

Markdown can be easily converted to structured data using javascript string manipulation.

const title = recipeText.split('~~~')[1].trim();
const Ingredients = recipeText.split('## Ingredients')[1].trim();
const Instructions = recipeText.split('## Instructions')[1].trim();

Conclusion

Given the JSON limitations and what feels like chatGPT's preference for markdown like output markdown becomes the path of least resistance and provides a more reliable and flexible alternative for structuring AI-generated content. It feels uncomfortable to use strings like this and is certainly not ideally as these characters are part of actual input so can conflict with data structure. In other words we aren't protected from a user inputting three tildes in a row causing our parsing to see a delimiter and split the string in a way we didn't intend. However, in practice this is more rare than the inconsistencies in JSON output which is certainly a surprise to me. Further, we do allow for all input to be edited by the user so if the parsing fails they can fix it themselves. Not ideal, but does allow us to make an actually product, not just a toy.

See it in action https://www.recipe2kitchen.com

AI-Powered Recipe Apps: Transforming the Cooking Experience

AI can revolutionize recipe apps by:

Generating complete recipes from minimal user input.
Extracting structured recipes from unstructured text (e.g., blog posts, PDFs, or handwritten notes).
Converting ingredients into shopping lists and suggesting meal plans.
Enhancing user experience with natural language processing (NLP) for search and recommendations.