Adding Google Search to my n8n-driven Job Search Agent

This week, I improved my AI Job Search Agent (implemented in n8n) to also include a Google Search for each of the job boards I’m querying – which allows me to sort by posting date and limit the time period.

To do this, I called the Google Custom Search API.

Calling the Google Search API

You can set up a free account that allows for 100 daily searches.

1. In the Google Cloud Console, add the Google Custom Search API and create an API Key for it (your ‘key’)
2. Create an a ‘custom search engine’ (CSE) on Google’s Programmable Search Engine – which gives you a Search Engine ID number (your ‘cx’)
3. Build your Google Custom Search Engine API call using the Google’s cse.list REST API
   • key – use your API key
   • cx – use your Search Engine ID number
   • siteSearch – the job board you want to limit your results to (e.g., greenhouse.io)
   • siteSearchFilter – use i to mark that you want to include the site listed in your siteSearch value
   • sort – I set the value of date to return the newest postings first; mitigating the issue of high-ranked prior results crowding out new results
   • dateRestrict – can be used to restrict the URLs returned based on date. I initially used this, but found that I lost a lot of valid search results and so removed it from my query filter.
   • q -and add your query

Additional information can be found on Google’s Introduction to the Custom Search JSON API page.

Adding the Google Search Call to n8n

With the API call crafted and tested in the API Explorer – let’s add it to our n8n workflow.

1. Add an HTTP Request node to the workflow
2. Rename and configure the properties of the node
   • Set Method to GET
   • Set URL to https://www.googleapis.com/customsearch/v1
   • Toggle Send Query Parameters to true/on
   • Add your Query Parameters (as discussed/tested above)
3. Wire your node input and output flows and test it out.

Note that I also added a Split Out node following each Google API call. But that’s it – it worked out pretty beautifully for me.

In the near future, I’m looking at doing the following tasks with the Google Search API call:

• Parse multiple pages. I implemented a do/while loop to retrieve all of the search results (rather than the first 10). For more information on this topic, see my tip on retrieving all google search results in n8n.
• Language filter. I’m occasionally seeing results for non-English roles, which I don’t need. However, the results are few and far between. I may play with the lr parameter in the future, if this becomes an issue.

Note – Mixing Google and Brave API Calls in a Single Workflow

One flag to call out is that the Google and Brave use different JSON schemas, which complicated my workflow at first. This created a bunch of spaghetti flows that annoyed me, until I added logic to check the JSON schema values. This is how I dealt with that:

First, I merged all of the Search Results together so that we can use a single loop to process all of the search results. I know that it looks ugly, but I’ll clean it up later. 😀

Second, I feed the results into a loop that calls a sub-workflow to process each result. This workflow contains the AI analysis and Airtable checks that I detailed in last week’s blog post.

And having the call out to a sub-workflow has been great for abstracting the JSON differences out. And this workflow call is where I handle the JSON schema differences. I do this using the .hasfield('value') method on the JSON item. The call looks like the following:

{{ $input.item.json.hasField('link') ? $json.link : $input.item.json.hasField('url') ? $json.url : "unknown" }}

The call itself uses an inline if statement, checking for the presence of the JSON field, and then using it if it is present. In the above, I check for the ‘link’ value (present in the Google API JSON) and then check for ‘url’ value (used in the Brave API JSON) if ‘link’ isn’t present. If neither are present, I pass ‘unknown.’

Third, I handle the results of the sub-workflow for reporting purposes and send out my status email with the various results.

For additional information on the larger workflow, check out the parent post on building an AI Job Search Agent with n8n.

I hope you found this useful.