Every integration with a state platform eventually has the same bad afternoon. The endpoint stops answering, the queue backs up, somebody starts pressing the button manually, and by evening the same invoice is in the system three times.
None of that is caused by the outage. It is caused by what the software does during one.
What "down" actually looks like
It is almost never a clean refusal. Clean refusals are easy. What you get instead, in rough order of frequency:
Timeouts. The connection opens and nothing comes back. Your client waits, then gives up, and you have no idea whether the request arrived. This is the dangerous one, and everything below is about it.
A 500, or a 502 from something in front of the application. Unambiguous, and the easiest to handle.
An HTML error page with a 200 status. The load balancer answers, the application behind it does not, and you get a courtesy page. If your code checks the status and then parses, this is the case that produces a nonsense error far from the cause. We have written about this exact failure in another context, because it is not specific to any one institution.
Slow but working. The worst for capacity planning. Requests succeed in ninety seconds instead of two. If your worker has a fixed timeout of thirty, you will generate a large number of "failed" submissions that in fact succeeded.
That last sentence is the whole article, so let me say it plainly: a timeout is not a failure. It is an unknown. Code that treats the two the same will submit duplicates, and duplicates in a fiscal system are considerably more annoying to unwind than a late filing.
Idempotency, which nobody gives you
The right answer to "did my request arrive" is an idempotency key: a value you generate, send with the request, and which the server uses to recognise a repeat. Plenty of modern APIs support this. Assume yours does not.
So you build the equivalent yourself, out of two rules.
One in-flight submission per document, enforced in your own database. Not in memory, not in the worker, in a row. Mark the invoice as submitting, with a timestamp, before the HTTP call. A second worker that finds that mark does not send.
Never retry a timeout blindly. Reconcile first. On an unknown outcome, do not resend. Query the platform for the state of that document. If the upload index exists, record it and stop. Only if the platform has no record of it do you send again.
That second rule is what turns "retry" from a hazard into a routine. It costs one extra request and it is the difference between an outage being an inconvenience and an outage being a cleanup job.
Backoff that respects a shared system
When a state platform is struggling, every integrator in the country is retrying against it. Retrying hard makes the outage worse for everybody including you.
Exponential backoff with jitter: wait a couple of minutes, then four, then eight, up to a ceiling of maybe an hour, with a random offset so that ten thousand systems do not all wake up on the same second. The jitter matters more than the curve. Without it, a synchronised retry storm forms every time the platform comes back, and it knocks the platform over again.
Cap the attempts, but cap them at a number of hours rather than a count. What you care about is "this has been failing for six hours", not "this has failed nine times".
The 25th problem
Romanian fiscal deadlines cluster. The platform is busiest exactly when everybody needs it, which is month end and the days around the 25th. Assume degraded performance in those windows and design so it does not matter.
The way to make it not matter is to stop submitting at the deadline. The transmission window is five working days from the invoice issue date, and the instinct is to treat that as a budget to be spent. It is not a budget, it is a buffer. Submit on issue. Then a two day outage is invisible, because you had three days of slack you never planned to use.
Teams that queue submissions for "the batch on Friday" are choosing to have no slack, and they discover this on the Friday that ANAF is unavailable.
Note also that five working days needs a Romanian public holiday calendar, and that calendar moves every year with Orthodox Easter. A counter that measures 120 hours is wrong, and it is wrong in the expensive direction precisely around Easter and early December.
What to alert on
Not errors. Errors during a known outage are noise, and after two hours of them people mute the channel, which is when the real problem arrives unobserved.
Alert on age: the oldest unsubmitted document in the queue is more than N hours old. That single check catches the outage, the crashed worker, the expired token, the misconfigured environment variable, and the case where somebody disabled the cron job in March. It is one alert and it means something.
Then alert separately, and loudly, on anything that has been in the queue for more than half of the legal window. That is not a technical alert. It is somebody needs to make a decision alert, and it should reach a person, not a channel.
The part that is not software
Have a manual route and make sure somebody has used it. The SPV portal accepts a file upload. If your system is down rather than the platform, and the deadline is tomorrow, the answer is one person with the XML file and twenty minutes, not an emergency deployment.
Print that procedure. Put it somewhere that does not require the system to be up in order to read it. It will be needed roughly once every two years, by whoever is available, who will not be the person who wrote the integration.
