Cloudflare WAF Migration
Incident Report for TCP Software
Resolved
This incident has been resolved.
Posted Sep 30, 2021 - 17:07 CDT
Update
A fix has been deployed and we are monitoring the results.
Posted Sep 29, 2021 - 05:22 CDT
Update
Hello OnDemand Customers!

In an effort to be proactive regarding potential service outages related to the Cloudflare migration, we want to give an update on one of the issues we have identified.

Customers that use IP restrictions to restrict clock operations may experience the following error when attempting to perform a clock operation via the TCP WebClock:

Unable to establish connection with provided credentials. Please try updating your software.

If your organization uses IP restrictions and experiences this issue, please reach out to TCP support as soon as possible so that a temporary fix can be put in place until we have a permanent fix ready for deployment (expected to be ready for release early next week).

You can contact TCP support using the following methods:
Chat or Email: https://www.tcpsoftware.com/contact#FeatureSupport - Available 24/7
Phone: 325-223-9300 - Available Monday - Friday 7 AM - 7 PM CST

We apologize for any inconvenience this error may cause. We are working diligently to prepare, test and deploy a permanent fix.
Posted Sep 24, 2021 - 15:15 CDT
Monitoring
We have completed the migration from Imperva to Cloudflare for our Web Application Firewall (WAF).

Initial tests indicate a successful migration.

We will continue monitoring the results overnight through tomorrow and provide updates on the migration as necessary.
Posted Sep 23, 2021 - 21:59 CDT
Update
Hello OnDemand Customers!

We wanted to provide some additional information on the WAF migration to Cloudflare and what that means for you.

The Web Application Firewall controls the inbound traffic to the TCP application servers (traffic from you to our servers). Outbound traffic (traffic from our servers back to you) is generally not affected by the WAF.

Should you whitelist the Cloudflare IP range?

While TCP recommends whitelisting the IP range, it is not required. We are recommending it as a proactive measure to prevent potential service disruptions. The majority of our customers will remain unaffected by this migration because they do not have any specific outbound communication rules set up on their network.

However, some customers choose to limit outbound traffic to the WAF IP range. For example: a network rule that limits outbound communication of physical clocks to the WAF IP range. In this example, whitelisting Cloudflare's IP range is required. Attempting to resolve to the Imperva IP range after the migration will result in no communication because the inbound traffic is no longer controlled by Imperva's Web Application Firewall.

For more information on Web Application Firewalls, please visit https://www.f5.com/services/resources/glossary/web-application-firewall
Posted Sep 07, 2021 - 16:02 CDT
Investigating
In an effort to continuously improve our service levels, TCP has made the decision to migrate to Cloudflare for WAF (Web Application Firewall) services.

TCP will begin routing all traffic through Cloudflare on September 23rd, 2021 at 9:00 PM CST.

We do not anticipate any service interruptions. However, if your organization uses network security rules to control traffic to and from TCP, please make all necessary adjustments to inbound and outbound security rules before September 23rd, 2021 at 9:00 PM CST.

The Cloudflare IP range list can be found here: https://www.cloudflare.com/ips/

Additional Information:

Cloudflare uses an anycast network. This means that domains can potentially resolve to any of the IP addresses listed on the IP range list posted above.

If your organization uses network security rules to control traffic to and from TCP, it is highly recommended that this IP range be whitelisted on your network to prevent any service disruptions.

TCP will continue to send out email reminders regarding the Cloudflare WAF migrations, and any relevant updates regarding the migration will be posted to this status page.
Posted Aug 30, 2021 - 07:30 CDT
This incident affected: TimeClock Plus Time and Attendance (Group1.tcplusondemand.com, Group2.tcplusondemand.com, Group3.tcplusondemand.com, Group4.tcplusondemand.com, Group5.tcplusondemand.com, Group10.tcplusondemand.com, Group10-2.tcplusondemand.com, Group12.tcplusondemand.com, Prod01.tcplusondemand.com, Prod02.tcplusondemand.com, Prod03.tcplusondemand.com, Prod04.tcplusondemand.com, Prod05.tcplusondemand.com, Prod06.tcplusondemand.com, Prod07.tcplusondemand.com, Prod08.tcplusondemand.com, Prod09.tcplusondemand.com, Prod10.tcplusondemand.com, Prod11.tcplusondemand.com, Prod12.tcplusondemand.com, Prod13.tcplusondemand.com, Prod14.tcplusondemand.com, Prod15.tcplusondemand.com, Prod16.tcplusondemand.com, Prod17.tcplusondemand.com, Prod18.tcplusondemand.com, Prod19.tcplusondemand.com, Prod20.tcplusondemand.com, Prod21.tcplusondemand.com, Prod22.tcplusondemand.com, Prod23.tcplusondemand.com, Prod24.tcplusondemand.com, Prod25.tcplusondemand.com, Prod26.tcplusondemand.com, Prod27.tcplusondemand.com, Prod28.tcplusondemand.com, Prod29.tcplusondemand.com, Prod30.tcplusondemand.com, Prod31.tcplusondemand.com, Prod32.tcplusondemand.com, Prod33.tcplusondemand.com, Prod34.tcplusondemand.com, Prod35.tcplusondemand.com, PHR1.tcplusondemand.com, PHR2.tcplusondemand.com, PHR3.tcplusondemand.com, PHR4.tcplusondemand.com, PHR5.tcplusondemand.com, PHR6.tcplusondemand.com, PHR7.tcplusondemand.com).