System status
Current health of every Phyto Nex AI component and connector, 90 days of uptime history and a written postmortem for every incident.
All systems operational
- 90-day uptime
- 99.986%
- Core platform
- 99.990%
- Incidents, 90 days
- 6
13 components monitored every 60 seconds from US East and Mumbai
Components
Hover any day to see its uptime. Contractual SLA: 99.9% on Mid-Market and Growth, 99.95% on Enterprise.
Web app
OperationalAudit engine
OperationalOCR pipeline
OperationalSlack agent
OperationalTeams agent
OperationalEmail delivery
OperationalSAP Concur
OperationalRamp
OperationalBrex
OperationalExpensify
OperationalNavan
OperationalNetSuite
OperationalQuickBooks Online
OperationalIncident history
Every incident gets a public postmortem within five business days: impact, root cause, fix and follow-ups.
Enterprise customers receive incident notices by email and in a shared Slack channel. Others can subscribe by writing to support@phytonexai.com.
Security and incident responseDelayed SAP Concur report ingestion
DegradedSAP Concur1h 47mResolved
Impact
New Concur expense reports for 38 tenants arrived in Phyto Nex AI up to 1 hour 47 minutes late. No data was lost and no flags were sent without AP approval.
Root cause
Concur applied a stricter per-company rate limit on the v4 expense reports endpoint during US month-end. Our backoff treated the new 429 responses as transient and retried too aggressively, extending the throttle window.
Resolution
We lowered per-tenant concurrency, switched to the Retry-After header for pacing and backfilled every delayed report in order.
Follow-ups
- Adaptive rate limiting per vendor endpoint (shipped Oct 3)
- Synthetic ingestion lag alert at 10 minutes instead of 30
- Joint review with SAP Concur partner engineering
14:12 UTC
Investigating increased ingestion lag for SAP Concur.
14:41 UTC
Identified rate limiting on the expense reports endpoint. Reducing concurrency.
15:26 UTC
Ingestion caught up for most tenants. Backfill in progress.
15:59 UTC
Resolved. All delayed reports processed.
Elevated OCR latency for multi-page folios
DegradedOCR pipeline52mResolved
Impact
Hotel folios over 4 pages took up to 9 minutes to extract instead of the usual p95 of 4.2 seconds. Single-page receipts were unaffected.
Root cause
A node pool autoscaling policy capped GPU capacity at the previous month's peak, and the September month-end spike from new customers exceeded it.
Resolution
Raised the capacity ceiling, drained the queue and moved scaling to queue depth rather than CPU utilization.
Follow-ups
- Month-end capacity forecast reviewed weekly
- Queue-depth autoscaling for all vision workers
05:03 UTC
Investigating slower extraction for long folios.
05:21 UTC
GPU capacity ceiling identified. Raising limits.
05:55 UTC
Resolved. Queue fully drained.
Microsoft Teams message delivery delays
DegradedTeams agent38mResolved
Impact
Approved pushback messages to Microsoft Teams were delivered up to 38 minutes late. Slack and email were unaffected.
Root cause
Microsoft Graph throttled proactive bot messages in the East US region during an upstream service incident.
Resolution
Messages queued safely and were delivered in order once throttling cleared. No messages were dropped or duplicated.
Follow-ups
- Optional email fallback after 15 minutes of Teams delay, configurable per tenant
16:40 UTC
Teams deliveries delayed due to upstream throttling.
17:18 UTC
Resolved. Queue delivered.
Partial outage of the AP review workbench
Partial outageWeb app23mResolved
Impact
About 31% of requests to the review workbench returned errors. Audits continued in the background and no findings were lost.
Root cause
A deploy included a database migration that locked the findings table longer than expected under production load.
Resolution
Rolled back the deploy, then reran the migration online in batches during low traffic.
Follow-ups
- Lock timeout guard on all migrations
- Canary deploys now hold at 5% for 15 minutes
13:07 UTC
Elevated error rates on the review workbench.
13:15 UTC
Rolling back the latest deploy.
13:30 UTC
Resolved. Error rates back to baseline.
NetSuite write-back failures for 3 tenants
DegradedNetSuite2h 10mResolved
Impact
Approved deductions and tax categories for 3 tenants did not sync to NetSuite. Nothing was written incorrectly; syncs were retried after the fix.
Root cause
A NetSuite role permission change on the customer side removed access to the custom record used for tax classification, and our error message did not name the missing permission.
Resolution
Worked with the three customers' admins to restore the permission and replayed the queued syncs.
Follow-ups
- Pre-flight permission check with a precise error in the admin console
09:02 UTC
Investigating NetSuite sync failures.
10:20 UTC
Root cause identified as a customer-side role change.
11:12 UTC
Resolved. Queued syncs replayed.
Delayed weekly digest emails
DegradedEmail delivery1h 05mResolved
Impact
Controller weekly digests were delivered up to 65 minutes late. Pushback emails were unaffected.
Root cause
Our transactional email provider deferred a batch due to a reputation check on a newly added sending IP.
Resolution
Moved digests back to the established IP pool and completed IP warm-up over two weeks.
Follow-ups
- Separate IP pools for digests and pushback messages
12:30 UTC
Digest emails delayed.
13:35 UTC
Resolved.
Stop playing expense police. Put policy enforcement on autopilot.
Connect Phyto Nex AI to your expense tool in 15 minutes and see what slipped past managers this month.
14-day free production trial. Read-only connectors. No card required.