Daily Reddit Scraper

This scheduled trigger runs once per day. Daily timing provides comprehensive coverage of new posts while avoiding excessive API calls.

This step fetches posts from a Reddit subreddit with sort_by "New" to get the most recent posts first.

Configuration: You must configure the subreddit URL or name during setup—specify which subreddit to monitor (like "r/ecommerce", "r/shopify", or whatever community is relevant to your domain). The sort_by parameter is set to "New" to prioritize recent posts where engagement is most valuable. The step returns an array of posts as {{reddit}} with each post containing title, description, url, author, score, and other metadata.

API considerations: Reddit's API may limit how many posts are returned per request. The workflow processes what's available, typically the most recent 25-100 posts depending on Reddit's response.

This loop step iterates through each post in the {{reddit}} array. For every post, the loop executes AI analysis, JSON cleaning, filtering, and Slack notification steps using {{loop.title}}, {{loop.description}}, {{loop.url}}, and other post properties. This enables post-by-post processing that handles large numbers of posts efficiently.

This step sends post data to AI with detailed classification instructions.

Critical configuration: You must customize the AI prompt by replacing placeholder text:

1. Replace "[INSERT YOUR DOMAIN / PRODUCT / AREA]" with your business domain (like "Shopify automation", "email marketing", "inventory management")
2. Replace "{{insert keywords here}}" with comma-separated keywords relevant to your business (like "workflow automation", "shopify integration", "order management")
3. Replace "[YOUR COMPANY / TEAM]" with your company name

The AI receives {{loop.title}}, {{loop.description}}, and {{loop.url}} and evaluates:

1. Is the post relevant to your domain using semantic analysis and keyword matching?
2. Does engagement make sense (is there a value proposition or conversation fit)?
3. Why is it relevant (1-2 sentence explanation)?
4. What integration or use-case ideas fit this discussion?

The AI returns structured JSON: {"url": string, "relevant": boolean, "makes_sense": boolean, "explanation": string, "matched_keywords": array, "integration_ideas": array, "confidence": float}.

Response format: The AI is instructed to return ONLY raw JSON without markdown formatting, but occasionally may wrap responses in code fences, which is why the next step cleans the output.

This custom JavaScript step processes the AI response to strip markdown formatting. The cleanJsonResponse function removes json and wrappers if present, trims whitespace, and returns clean JSON. This enables reliable parsing in subsequent steps even when AI occasionally ignores the "no markdown" instruction. The cleaned JSON is returned as {{custom}} with properties like {{custom.relevant}}, {{custom.explanation}}, {{custom.matched_keywords}}, and {{custom.integration_ideas}}.

This filter checks if {{custom.relevant}} equals "true". If the AI determined the post is relevant to your domain and keywords, the workflow proceeds to send a Slack notification. If not relevant, the workflow skips notification and moves to the next post in the loop. This filtering prevents your team from being overwhelmed with irrelevant Reddit posts.

This step posts a formatted message to Slack.

Configuration: You must select your Slack channel during setup—first invite the MESA Slack app to your desired channel by typing @MESA and clicking Invite, then select that channel. The message uses a template with:

1. Post URL ({{loop.url}})
2. AI's relevance explanation ({{custom.explanation}})
3. Post description ({{loop.description}})
4. Matched keywords list ({{custom.matched_keywords}})
5. Integration ideas list ({{custom.integration_ideas}})

The markdown_text uses Liquid templating with conditional logic to only display keyword and integration sections if arrays contain data, preventing empty sections in notifications.

Message format: The template creates structured Slack messages with emoji, bold headers, and bullet lists that make information scannable for quick assessment.