How to bulk validate Airtable data to save API checks

To dramatically improve the performance and cost of my n8n job search workflow, I updated the workflow to bulk validate my Airtable data to save API checks. This post shares how I accomplished this within my n8n service using standard nodes.

Challenge: As workflow ran against a collection of search results each day, the workflow called Airtable to validate if the URLs had been seen before. This meant that the Airtable lookup API was being called 90 times per day for the same URL – wasting compute (processing previously processed results) and wasting Airtable API calls (particularly important for my free tier account).

The process was surprisingly easy using the n8n Merge node:

• Airtable cache: I did a bulk Search using the Airtable node, pulling the entire table
• Anti-Merge filter: I used the Merge node to do a Combine that keeps only non-matches

Practically, I added the anti-merge after combining the returned search results, matching my Job Description Link column in Airtable against the link value in the Google search results.

As you can see in the image above, the results are fantastic. For today’s execution, I filtered the day’s 92 search results (24 from Google + 68 from Brave) down to 5 results that aren’t in the Airtable cache. These 5 results will now go on for further processing – saving me 86 Airtable API calls for today’s workflow execution. Yay!

Additional Notes:

• The workflow may end at the merge step if all of the results filter out – meaning that no confirmation email is sent at the end to signal that the workflow ran successfully. To solve this, I tweaked the Merge nodes to toggle the Always Output Data option so that the workflow progresses and ends appropriately.
• To limit the amount of data that I’m pulling down and storing, I’m playing with the Output Fields option in the Airtable node to limit the number of output fields / columns that I am retrieving from Airtable.

With the workflow updated, I’m going to keep an eye on my Airtable API usage, hopefully seeing that go down. I’ll update this post as I learn more, but I hope that this tip helps others as much as it helped me. Happy flowgramming!