A year of AI-run ads: what held up, what we dropped

A year ago the question about letting software run live ad accounts was how much it could do. Twelve months later that turns out to have been the wrong question. The one that matters is how much it should be allowed to do, and how you find out when it is wrong.

This is the retrospective. What held up, what got dropped, four things that turned out truer than expected, four mistakes and what each cost, and the parts still held loosely. Everything here is written so you can apply it to your own account rather than admire ours.

There are no business result figures in this article. The reason is in the questions section, and it is not that the results were poor.

What changed most in a year

The centre of gravity moved from capability to authority.

The first half of the year went into making the system able to do more things. By the end, most of the work was in defining precisely what it is permitted to do, within which bounds, and what trace it leaves behind. Adding another action to the library stopped being interesting somewhere around month five. Making every action state its own limits turned out to be the whole job.

If you are somewhere in that first phase now, the shortcut is: stop counting what the tool can do and start writing down what it may do, on this account, with a ceiling attached. That list is short, and it is the only one anybody reads twelve months later.

Five decisions that held up

Permission per action, not per system. Each action carries its own authority level rather than the whole thing being on or off. This was the best decision of the year and it was not the original one. The first version had a three-mode global switch, and every single person who reached the automatic mode wanted it for two actions and not the other eighty.

What it means for you: if a tool asks you to trust it as a whole, it is asking the wrong question. Trust attaches to a specific action with a specific worst case, and you should be able to grant it that narrowly.

The practical version of this is a sentence you should be able to finish about any action you have switched on: "if this goes wrong tonight, the worst it can do is ___." If the blank needs a paragraph, the permission is too broad. If you cannot fill it in at all, the tool is not telling you enough.

Advisory before execute. No action gets execute rights until it has run in advisory long enough to produce evidence on that specific account. This creates friction in sales, because people want it working on day one. Every early grant we watched produced an unpleasant surprise inside a few weeks.

Four weeks is the number worth holding to. Long enough to include a bad week, a promotion and a slow week, which is what you are actually testing for.

The advisory period only works if somebody records a verdict each time a rule fires. Agreed, disagreed, unclear — three options, one line. At the end of four weeks that record is the evidence, and it takes about thirty seconds a day to keep. Without it you arrive at the decision point with an impression rather than a count, and impressions favour whichever week ended most recently.

The six-field log. When, which rule, which entity, the numbers at the time, the change from what to what, and who authorised it. Three of those six were added after incidents rather than designed in, which is the honest version of that story.

The field that does the most work is the numbers at the time. Ad platform data changes retroactively — conversions attribute days late, spend gets restated — so a line that does not carry the values as they stood will make every action look premature when you read it back later. That single misreading is behind most of the cases where somebody switches off a rule that was doing its job.

Sequential writes. Slower, and what the slowness buys is a gap between actions. In that gap a guard can check, a limit can bind, and a person can stop the run. A system that changes forty things in one second invites suspicion even when all forty were right.

No unattended creative. There is no configuration in which copy gets written and published without a person. That is a stated position rather than a technical limitation, and it is the one we get asked to relax most often. The reason it does not move is in the reversibility levels: you can pull a creative, you cannot pull the impression.

What we dropped, and why each sounded right

The global mode. Three modes, easy to explain, easy to build. It assumed trust is a property of the system when it is a property of each action. Replaced by a permission table with one row per action, which cost a rewrite of the whole permission layer plus migrating every configuration people had already set.

Features gated by plan. Removed entirely in July 2026. For each gate, the honest answer to "why is this restricted?" was "because it has to sit on some tier", which is not a reason a customer should accept. It cost revenue in the short term, and that was the expected outcome rather than a surprise.

Optional bounds. Originally an action could be granted execute rights without stating an absolute ceiling. After one account's budget climbed steadily to a number nobody had chosen, bounds became a required field. Anything that grows by a percentage needs an absolute ceiling, because a percentage has no natural stopping point.

Automatic retries. Sounds convenient. In practice it is how one intended change becomes three. Every attempt is now written to the log with its error, and nothing repeats itself without a person.

Targets counted in articles. On the content side, "thirty articles a month" produced writing to hit a number. Replaced by coverage of keyword clusters, which is harder to report on and measures the thing you actually wanted.

Read those five together and they share a shape. Every decision that held up added a limit. Every one that had to be reversed removed a limit for convenience. The convenience gets collected in small amounts on ordinary days; the cost arrives all at once on the day that matters. When in doubt, add the limit. You can remove it later with evidence in hand, calmly, rather than during an incident.

Four things that turned out truer than expected

Most of the value is arithmetic, not the model

The hardest thing here to say, and the truest. Most of what the system surfaces comes from numeric comparisons a competent analyst could write as spreadsheet formulas. Spend past three times target cost per acquisition. Click-through rate down thirty percent week on week. Frequency past a threshold.

The model call adds three things on top: interpretation where arithmetic cannot conclude, prioritisation across findings, and the ability to answer a follow-up question in plain language.

Whether those three justify their cost depends entirely on your situation. A team with an experienced analyst who has time gets less from it. A team where the person managing ads also manages three other things gets a great deal more. We say this plainly because the opposite implication, that the model does the work, sets people up to expect the wrong thing and be disappointed by the right one.

Defensive actions are enough for most accounts

The prediction was that everyone would want proactive optimisation. What actually happens is that most accounts grant execute rights to one or two actions, both defensive, and stop there.

That is the correct configuration, not an over-cautious one. Stopping wasted spend when nobody is at a desk is the single largest piece of value available, and it requires granting nothing risky at all.

A good log increases delegation

The initial intuition was the opposite: a detailed log makes risk visible, so people will pull back. What actually stops people is not the fear that software will make a mistake. Everyone accepts it will, occasionally. It is the fear of a mistake they will not learn about.

When every action leaves a readable trace, that fear goes and what remains is calculable risk: how large a change, how many entities, how long before somebody sees it. Three numbers, which together make an ordinary business decision.

Slower earns more trust

Sequential execution looks inefficient on a diagram. After about two weeks the perception inverts, and it usually inverts on a specific day: the first time a run halts partway through and the person realises the halt saved them something.

Four mistakes and what each cost

Believing in a global mode. Cost: rewriting the entire permission layer, and migrating every configuration people had already set. The decision sounded eminently sensible when it was made. It just assumed the wrong thing about where trust lives.

Making bounds optional. Cost: one account where a well-performing campaign's budget climbed to a level nobody had consciously chosen. Performance held the whole way up, and the owner's objection was still correct: the arithmetic had decided, not a person.

Check this on your own account today. For every rule that can increase anything, find the absolute ceiling. If there is no number, that is the finding.

Shipping a log missing three fields. Cost: during the first incident review, reconstructing who had authorised what meant reading configuration history. An afternoon for a question that should have taken five minutes. The three missing fields were the prior value, the grant identifier, and the record of failed attempts.

Publishing while indexing was broken. The most expensive of the four. Several months of publishing on schedule while the index rate sat below a third, which means most of that writing went into pages nobody could reach.

The root cause was a hosting incident lasting a few hours that left the sitemap unfetchable, with nothing anywhere to signal it. The fix became one line on the incident checklist: after any outage, check the sitemap report. Ten seconds, and it closes a two-month gap.

If you publish on a schedule, add the same line. Any incident that touched the web server, however brief, gets a sitemap check afterwards.

What the content side taught us

Optimising old posts beats writing new ones. A new article starts at zero on four fronts: not crawled, not indexed, no inbound links, and no history telling the search engine what it covers. An older article sitting at position twelve already has all four. Almost nobody does this work, because editing old posts produces no feeling of progress. At month end, three new articles are something to show. Twelve improved old ones are not.

Structure matters as much as the writing. An article where no paragraph stands on its own will not get quoted, will not appear in an AI summary, and will not become a featured snippet. Fixing structure takes about twenty minutes an article and has the best return per minute of anything in a content operation.

Translating between languages does not work. Keyword clusters do not map one to one across languages. The concrete case: an English phrase around ad automation has clear search volume, and its literal Vietnamese equivalent returns nothing from autocomplete, meaning effectively nobody types it. Vietnamese speakers search for something closer to "AI running ads". Translate directly and the entire Vietnamese set targets phrases nobody uses, with nothing in any dashboard to tell you.

Good positions on the wrong cluster deliver nothing. We had a cluster ranking between positions five and twelve with healthy metrics, and dropped it entirely. Every query in it contained the word "official" — people were looking for the platform's own documentation, not a third party writing about it. Read the intent behind a query, not only the volume next to it.

Three things we still hold loosely

The read/write boundary may be too coarse. Every write runs sequentially. Some writes are genuinely harmless, though — applying a label, writing a note — and forcing those through sequential execution costs time and protects nothing. A finer classification would be better. It has not been built because the boundary is hard to draw without it drifting over time, and a drifting safety boundary is worse than a crude fixed one.

Ten saved versions is an arbitrary number. It came from an observation about how long configuration problems take to surface, not from anything principled. Twenty would probably be better and would cost almost nothing.

The builder palette may be too small. Seven condition types and ten action types on the ads side. The argument for keeping it small is that anybody can read a sequence somebody else built, and that argument is real. The cost is that some legitimate cases cannot be expressed and get handled outside the system, where there is no log and no bounds at all.

A representative account over twelve months

Composited from several accounts rather than describing one customer, and carrying no result figures.

Months one and two: watching. Measurement monitoring and basic leak protection, everything advisory. The first week produces seven recommendations. Three are old campaigns everyone knew were poor but nobody had switched off. Two are new campaigns still learning, and therefore wrong. Two are genuine finds. Adding an exclusion for campaigns under seven days old drops the rate to about three a week, all of them agreed with.

Month three: the first grant. Exactly one action: pause dead campaigns. Capped at two campaigns per run, with brand and new campaigns excluded. This is usually the month somebody notices the system doing real work, not because it is clever but because it acts at two on a Sunday morning.

Months four to six: careful expansion. Stepped budget increases added in advisory mode. This is where several campaigns turn out to have no absolute ceiling, which gets fixed before anything runs. Four rules that never fired get switched off. Removing is as valuable as adding and takes ten minutes.

Months seven to nine: the plateau. The phase nobody plans for. Most accounts stop here, and not because the system has run out of things to do. Everything remaining belongs to categories where granting execute rights is not worth it: structural changes, audience changes, creative.

Teams often read this as a failure of adoption. It is not. What stays unautomated stays unautomated for good reasons — the actions carry strategic weight, their failure modes cannot be undone, or the right answer depends on business context the software cannot see. Every account we have seen push past that boundary has walked it back. The useful line for that meeting: the goal was never to automate everything, it was to remove the parts that are repetitive, threshold-driven, and happen when nobody is watching.

Months ten to twelve: maintenance. Reading the morning summary, one quarterly review, seasonal threshold adjustments. This is the correct end state rather than stagnation. A system producing something that needs attention every single day has thresholds set too loosely.

What the permission table looks like after a year

Permission per action is the decision held most firmly, so here is what it actually produces. One row per action, four columns: the action, the level, the bounds, and who granted it with a date.

On a settled account most rows read the same way. Around half the library sits at level zero, not referenced at all, because those actions apply to platforms the account does not run, objectives it does not use, or structures it does not have. That is a configuration choice rather than a failure to configure, and it is worth saying out loud in a review before somebody counts unused rows as waste.

Nine to twelve rows sit at advisory. These are the ones producing the morning summary, and most of them will stay advisory permanently: anything touching creative, anything with a strategic dimension, anything where the right answer depends on business context the software cannot see.

Two rows say execute, with bounds written out in full: maximum entities per run, maximum change size, absolute ceiling, cooldown, exclusion list. Both are defensive.

The whole table fits on one screen and can be read aloud in a meeting. That was the design target and it turned out to be the right one. A permission model nobody can read is a permission model nobody reviews.

What a quarterly review actually does to it

Three kinds of change, in rough order of frequency.

Rows moving to level zero. Actions that never fired in three months. This is the most common outcome and the most useful, because a shorter list is a list people can hold in their heads.

Bounds tightening. Usually seasonal. A ceiling set during a busy period no longer fits the quarter you are in.

Occasionally, a revocation. An execute grant going back to advisory because the account structure changed and the evidence behind the original grant no longer applies. Normal outcome, not a failure.

What almost never happens at a review is a new execute grant. Those come from evidence accumulating in advisory mode, which happens on its own timeline rather than on the calendar.

Three questions clients ask back at us

So what does it actually save?

Not ad spend. That depends far more on product, price and market than on anything a rule can do.

What it saves is time and latency. Time in not opening reports each morning hunting for anomalies. Latency in the gap between a problem appearing and somebody knowing about it, which goes from days to under a day.

On a small single account those two may not justify the cost. For somebody managing several accounts they are the entire value, and the calculation is usually obvious within a month.

If most of it is arithmetic, why not use the platforms' own rules?

For simple conditions inside one platform, you should. Google's and Meta's automated rules are free and they run natively, which is two real advantages.

Three things they do not do: conditions that span platforms, thresholds derived from the account's own history rather than a number you typed, and a record tying a change to the permission that authorised it. If everything you need fits inside one platform, use its tools and do not pay for a layer that adds nothing.

A year in and it still is not fully autonomous?

No, and that is not the destination. Some things should not run unattended at any level of trust, not because the software performs badly but because being wrong cannot be undone. Content a customer has already seen is the clearest example.

A fully autonomous system is one that has skipped the question about reversibility levels rather than answered it.

Four questions we still cannot answer

How much will this save me?

Not answerable, and anybody who gives you a figure should be asked where it came from. The result depends on how much waste already exists in the account, and that varies enormously. A tightly managed account has almost nothing to cut. One neglected for six months has a great deal. The only defensible statement is that it finds what is already there.

What is the payback period?

Same problem, with another layer on top: most of the value is time saved, and converting time into money is a calculation everyone does differently and nobody does consistently.

Do you have a case study for a business like mine?

There are real cases, and they are not packaged with growth figures, because separating the system's effect from everything else the team did in the same period is not something anybody can do honestly. What can be described is the process: which rule fired, why, what was changed afterwards. That part is checkable, and it is the part somebody can learn from.

Do my competitors use it?

Not answerable, and not a basis for a decision even if it were.

Three things that did not change all year

Fix measurement before optimising. No exceptions. Optimising against wrong numbers drifts in a fixed direction, so it never self-corrects, and the reports look fine the entire time.

Defensive before growth. Stopping something wasteful costs a few hours of delivery when it is wrong. Pushing money toward something that looks good costs money quietly, and it looks like success while it is happening.

Record your judgement at the time. Not from memory three weeks later. Recollection of whether you agreed with a decision shifts remarkably fast once you know how it turned out, which is why a crude five-column spreadsheet is more useful than it looks.

If you are evaluating this category

Three questions to ask yourself before asking any vendor anything.

Are you doing repetitive work against clear thresholds? If so, that is the automatable part, and it is worth being specific about which tasks. If your work is mostly market judgement, there is much less here for you.

Is your measurement correct? If not, fix that first. Every rule reads platform numbers, and wrong numbers produce wrong rules very convincingly. Reconcile last month's orders in your own sales system against what the platforms reported. If the gap is beyond thirty percent, everything else waits behind this.

Will anyone read the morning summary? If not, then however good the tool is, it is another inbox nobody opens. This is the question that decides the outcome more often than the feature comparison does.

Three yeses and it is worth trying. Any no, and that no is the thing to fix first.

Run the trial properly if you run one. Four weeks, everything advisory, one person who reads the summary each morning and writes down whether they agreed. Do not grant execute rights during the trial, because a trial with execute rights is measuring how nervous you are rather than how good the rules are. At the end you will have between fifteen and thirty recorded verdicts, and that is enough to decide which one or two actions get to act on their own.

Then four questions for the vendor, in one sitting. Show me a real log line from a real account, and let me count the fields. Does it record the value before the change. What gets written when a platform call fails. Do unattended actions reference the permission that allowed them.

Frequently asked questions

What would you do differently starting over?

Three things. Make bounds a required field from day one. Build the six-field log before building the first action, because advisory mode is only useful if you can read what would have happened, threshold tuning needs to know what fired and whether you agreed, and the permission model is unverifiable without a record of grants. And fix indexing before publishing anything. All three cost nothing extra at the start and several times as much retrofitted.

Does any account run fully unattended?

No, and that is not the goal. The most common configuration is two actions executing, nine to twelve advisory, and the rest switched off.

How large a team does this need?

One person, provided that person reads the morning summary. The requirement is not headcount. A team of ten where nobody opens it is worse off than one person who opens it daily.

What gets misunderstood most often?

That granting execute rights removes the safety gates. It does not. Every gate applies in both modes. What changes is who performs the action and how quickly.

Why are there no business result figures?

Because separating the effect of the system from seasonality, competition and everything else the team did in the same period is not something we can do honestly. There is no control group. A number here would read better and it would be invented.

Should I wait for the technology to mature?

The parts most worth doing do not depend on how mature the technology is. Correct measurement, sensible thresholds and a readable log are right today and will still be right in two years. Waiting only delays accumulating your own data, which is the input nobody can give you.

Will better models change the answer?

Not on the part that carries most of the value. "Spend exceeded three times target" is a comparison, not a question requiring inference, and a better model does not make a comparison more correct. Where better models help is interpretation and follow-up questions, which is real but is not where most of the work happens.

What is the advice for someone starting now?

Switch on measurement monitoring first. Then two defensive actions in advisory mode. Record whether you agreed each time one of them fired. After four weeks, read that record and only then decide what to grant. That is the whole recommendation, and it has not changed in a year.

The year produced one conclusion nobody at the start would have predicted: what convinces people to grant more rights is not a long run of correct decisions. It is the first time the system was wrong and they found out immediately. An unbroken run of correct decisions says nothing about what happens when something goes wrong. One wrong action, caught within hours, with a log complete enough to explain why, fixed by adding an exclusion — that single episode demonstrates the whole safety apparatus working, and the conversation afterwards changes character.

So do not fear the first wrong action, and do not respond to it by switching everything off. One visible mistake is worth more than three months of smooth running that nobody audited.

If you take three things away, take these. Open your current tool's log and count how many of the six fields are present, which takes five minutes. For every execute grant you have active, write down its reversibility level, one line each, and revisit anything that lands at "not reversible" today. And reconcile last month's order count in your sales system against what the platforms reported, which takes fifteen minutes and decides what you should be working on next.

Further reading: advisory versus execute: a permission model, six fields every ad change log needs, and 19 starter rules and what each is for.

Start at orova.vn.