OpenAI recently released a tool called Operator, for existing subscribers to the ChatGPT Pro plan. Operator is described as “A research preview of an agent that can use its own browser to perform tasks for you.” In other words, a test of ChatGPT as an agent that can actually use a web browser to complete tasks for the user. 

This of course is a big step forward for ChatGPT which otherwise is limited in functionality to largely just communicating with users through the ChatGPT app. Operator seems comparable to the Anthropic Computer Use functionality, however Anthropic’s computer use tool is only available through the API. Operator is actually a web based application that any user can run simply through the browser. 

The Operator tool today is described as a “research preview” – more or less language from OpenAI indicating the product is still in beta. Certainly this was our experience in testing it, as the tool is rather buggy at the moment, but offers a window into the future of AI Agents and how we might use them for business tasks. 

What we Tried with Operator + Google Ads + Google Analytics

Day 1: Complete Failure

I signed up for Operator the first day of the release. The initial experience was very buggy. The agent was incredibly slow to the point of being unresponsive. After a long waiting period I was able to get it to login to Google Ads, and I tried to direct Operator to do some analysis of a Google Ads account. The response was simply “No” from Operator and given that it took an hour or more to even get here I gave up for the day.  

Day 2: Trying Google Analytics Tasks

Given my lack of success with Google Ads the prior day, I thought I’d try a Google Analytics task instead. I gave the agent a few simple prompts – to extract data from Google Analytics, migrate it into a Google Sheet, and make a chart. Firstly by day 2 the interface was much snappier and OpenAI had resolved the issue of overall responsiveness to some degree – though it is still a notably non-polished experience. 

The agent was actually largely successful at the task I assigned it – which was an easy task – of simply copying data from GA to GSheets and making a chart. It make some formatting and data errors, which I did not bother to try correcting as I felt it was a sufficient test. 

It’s clear that assuming Operator progresses forward roughly at the same pace as AI in general has progressed over the last 12 months, within a few months Operator will be able to very effectively use Google Analytics and perform GA analysis tasks in Google Sheets. (Though for right now, my Day 2 experience shows this to be a proof of concept – the tool remains too slow and buggy for real work, at least based on this test.) 

Day 3: Trying Again with a More Complex Google Ads Task

With a bit more confidence given my Day 2 experience, I tried again to get Operator to perform a Google Ads task. This time I was much more successful. Here’s the prompt I offered the Operator, admittedly a complex task, 

Operator actually made it a few steps into this prompt just fine – it was able to find the search query report in Google Ads, and export it to a Google Sheet. It did add a relevance column and it did also crawl the website I requested. However it was not very good at placing the relevance column in the sheet and I had to take control and fix a variety of Google Sheets mistakes it made. 

Once I fixed the sheet manually, Operator was actually able to do the query scoring task I assigned it, and it did so reasonably well for about 30 rows of data. The output below was created entirely by Operator. However it crashed around row 30 and got stuck on some loop within Google Sheets that it could not get over despite several attempts for me to fix it manually. 

The screenshot below shows where it got stuck. I tried many different prompts to get it out this error but for whatever reason it got stuck trying to deselect parts of the sheet and could not continue the prompt. 

Conclusion

This is just a few examples of different things I tried. I admit to being far from a power user of Operator so I am sure I could improve my prompting and direction. Generally speaking, right now I agree with OpenAI that this is at a “research preview” stage of development, at least for the tasks I was trying to complete. However, it is obvious that Operator succeeds as a proof of concept for what an AI agent could do once further developed. If Operator were 10X better, a reasonable expectation within let’s say 12 months, and possibly much less, it would indeed be a powerful tool for implementing any browser-based task with tremendous efficiency. We’ll see how fast OpenAI and others can create a single order of magnitude improvement in Operators. OpenMoves will continue testing and tinkering in the meantime.