Active Incident

Updated 3 minutes ago

Performance Issue on Cluster 1Degraded Performance - SEV 2

Incident Status

Degraded Performance - SEV 2

Components

Workfront Status

Locations

CL01



February 21, 2017 9:33AM MST
February 21, 2017 4:33PM UTC
[Investigating] The Workfront service is experiencing a performance issue. Users may see slow page loads and timeout errors when using the service. We are working to resolve the situation as quickly as possible.

Scheduled Maintenance

ProofHQ Scheduled MaintenanceScheduled Maintenance

Schedule

February 19, 2017 12:00AM - 2:00AM MST
February 19, 2017 7:00AM - 9:00AM UTC

Components

Data centers, ProofHQ Status

Locations

PHQ API, PHQ Email, PHQ Proof Processing, PHQ Web Application, PHQ Website, Proof1, Proof2

Description

Our hosting provider will be performing works at their end and so we are expecting intermittent performance issues during the maintenance window.

Schedule

February 19, 2017 12:00AM - 2:00AM MST
February 19, 2017 7:00AM - 9:00AM UTC

Components

Workfront Status

Locations

CL01, CL02, CL03, CL04

Description

Workfront's proofing service may be briefly affected during the scheduled maintenance.

Workfront Status




Degraded Performance - SEV 2

ProofHQ Status




Operational

Data centers




Operational

Locations

External Services

AWS EC2

AWS S3

Rackspace

ProofHQ Uptime

100.00%




Workfront Uptime

100.00%




Avg. Response Time

90.61ms




Avg. Round Trip

174.61ms




Total Requests

989499.33 Hits




ProofHQ Uptime

100.00%




Workfront Uptime

100.00%




Avg. Response Time

95.98ms




Avg. Round Trip

177.34ms




Total Requests

900704.21 Hits




ProofHQ Uptime

100.00%




Workfront Uptime

99.98%




Avg. Response Time

89.53ms




Avg. Round Trip

168.17ms




Total Requests

923520.87 Hits




Maintenance Windows North America: CL01 | CL02 | CL03 | ProofHQ 2nd and 4th Saturday 8:00 pm to 12:00 am MT 1st and 3rd Thursday 8:00 pm to 12:00 am MT EMEA: CL04 1st and 3rd Thursday 2:00 am to 4:00 am GMT CL01, CL02, CL03 Preview environments refresh on Saturday CL04 Preview environment refreshes on Friday **We infrequently schedule maintenance in the windows listed above. If down time is necessary, it will be scheduled in the corresponding window and announced on this page.**

History (Last 21 days)

Scheduled CL03 maintenance.Scheduled Maintenance

Description

CL03 will be temporarily offline for scheduled database maintenance.


Components

Workfront Status


Locations

CL03


Schedule

February 16, 2017 8:00PM - February 17, 2017 12:00AM MST
February 17, 2017 3:00AM - February 17, 2017 7:00AM UTC



February 16, 2017 8:00PM MST
February 17, 2017 3:00AM UTC
[Update] started

February 17, 2017 12:00AM MST
February 17, 2017 7:00AM UTC
[Update] finished
Performance Issue on Cluster 4Degraded Performance - SEV 2

Incident Status

Degraded Performance - SEV 2


Components

Workfront Status


Locations

CL04




February 16, 2017 3:15AM MST
February 16, 2017 10:15AM UTC
[Investigating] We are currently experiencing a performance issue on CL04 and are investigating. Correction: Impact ONLY on CL04

February 16, 2017 4:14AM MST
February 16, 2017 11:14AM UTC
[Investigating] CL04 performance has been improved. We are still in investigation process for the fix.

February 16, 2017 5:00AM MST
February 16, 2017 12:00PM UTC
[Investigating] We are still experiencing a performance issue on CL04 and are in urgent investigation process.

February 16, 2017 6:05AM MST
February 16, 2017 1:05PM UTC
[Resolved] Performance has stabilized on Cluster 4. Workfront continues to investigate the root cause.
Scheduled CL04 maintenance.Scheduled Maintenance

Description

CL04 will be temporarily offline for brief scheduled database maintenance.


Components

Workfront Status


Locations

CL04


Schedule

February 15, 2017 7:00PM - February 15, 2017 7:15PM MST
February 16, 2017 2:00AM - February 16, 2017 2:15AM UTC



February 15, 2017 7:00PM MST
February 16, 2017 2:00AM UTC
[Update] Started

February 15, 2017 7:15PM MST
February 16, 2017 2:15AM UTC
[Update] Finished

Incident Status

Operational


Components

ProofHQ Status


Locations

PHQ Proof Processing




February 15, 2017 2:35AM MST
February 15, 2017 9:35AM UTC
[Investigating] We are currently experiencing delays with static proofs processing due to high volume of submissions.

February 15, 2017 3:19AM MST
February 15, 2017 10:19AM UTC
[Resolved] We've adjusted the processing tools to eliminate issues and improve processing times. All queued proofs should be processed now. The processing times are back to normal.
CL04 database disruptionService Disruption - SEV 1

Incident Status

Service Disruption - SEV 1


Components

Workfront Status


Locations

CL04




February 12, 2017 5:25PM MST
February 13, 2017 12:25AM UTC
[Investigating] The Workfront service is currently unavailable. We are investigating the cause of the disruption and will provide more information as it becomes available.

February 12, 2017 5:40PM MST
February 13, 2017 12:40AM UTC
[Resolved] The Workfront service has been restored. The disruption was caused by a temporary failure in the database.
Scheduled CL02 maintenance.Scheduled Maintenance

Description

CL02 will be temporarily offline for scheduled database maintenance.


Components

Workfront Status


Locations

CL02


Schedule

February 11, 2017 8:00PM - February 12, 2017 12:00AM MST
February 12, 2017 3:00AM - February 12, 2017 7:00AM UTC



February 11, 2017 8:00PM MST
February 12, 2017 3:00AM UTC
[Update] Started

February 12, 2017 12:00AM MST
February 12, 2017 7:00AM UTC
[Update] Finished
Network Failure in primary datacenter Service Disruption - SEV 1

Incident Status

Service Disruption - SEV 1


Components

Data centers, Workfront Status


Locations

CL01, CL02, CL03, DC1




February 7, 2017 4:16PM MST
February 7, 2017 11:16PM UTC
[Investigating] An issue has occurred with the networking equipment in the primary datacenter

February 7, 2017 4:29PM MST
February 7, 2017 11:29PM UTC
[Resolved] The networking issue has been resolved.
Performance ImpactDegraded Performance - SEV 2

Incident Status

Degraded Performance - SEV 2


Components

Workfront Status


Locations

CL01




February 7, 2017 8:45AM MST
February 7, 2017 3:45PM UTC
[Investigating] We are currently experiencing a performance issue and are investigating.

February 7, 2017 9:25AM MST
February 7, 2017 4:25PM UTC
[Resolved] The issue has been resolved and performance has returned to normal.