Resolved -
This incident was fully resolved on Monday 16 November when all task processing queues returned to nominal levels.
Nov 16, 19:00 AEDT
Monitoring -
A fix has been implemented and we are monitoring the results. The Funnelback Admin console is expected to be operating normally, with the exception that some informational data may be to 1 minute out of date. Collection updates have begun to execute and long queue times are expected until the backlog has been cleared. Please don't manually add updates to the task queue, your tasks will be processed.
Nov 16, 13:38 AEDT
Identified -
Squiz is currently experiencing a service degradation affecting the Funnelback Admin console and background task processing for customers in the Australian region. Users may experience slowness, delayed collection updates, or intermittent 502/504 errors when accessing the Funnelback Admin interface.
Our engineering teams have identified the source of the elevated server load and are actively implementing mitigations to restore normal performance.
Nov 16, 12:04 AEDT
Update -
We are continuing to monitor and rectify for any further issues.
Nov 16, 11:34 AEDT
Update -
We are continuing to monitor for any further issues.
Nov 13, 16:01 AEDT
Monitoring -
A fix has been implemented and we are monitoring the results. The Funnelback Admin console is expected to be operating normally. Collection updates have begun to execute and long queue times are expected until the backlog has been cleared.
Nov 13, 15:04 AEDT
Update -
We are continuing to work on a fix for this issue.
Nov 13, 14:35 AEDT
Update -
We are continuing to work on a fix for this issue.
Nov 13, 14:11 AEDT
Identified -
Squiz is currently experiencing a service degradation affecting the Funnelback Admin console and background task processing for customers in the Australian region. Users may experience slowness, delayed collection updates, or intermittent 502/504 errors when accessing the Funnelback Admin interface.
Our engineering teams have identified the source of the elevated server load and are actively implementing mitigations to restore normal performance.
Nov 13, 13:53 AEDT