Time and Attendance On Demand.
Cloudflare WAF Migration
Update - Hello OnDemand Customers!

We wanted to provide some additional information on the WAF migration to Cloudflare and what that means for you.

The Web Application Firewall controls the inbound traffic to the TCP application servers (traffic from you to our servers). Outbound traffic (traffic from our servers back to you) is generally not affected by the WAF.

Should you whitelist the Cloudflare IP range?

While TCP recommends whitelisting the IP range, it is not required. We are recommending it as a proactive measure to prevent potential service disruptions. The majority of our customers will remain unaffected by this migration because they do not have any specific outbound communication rules set up on their network.

However, some customers choose to limit outbound traffic to the WAF IP range. For example: a network rule that limits outbound communication of physical clocks to the WAF IP range. In this example, whitelisting Cloudflare's IP range is required. Attempting to resolve to the Imperva IP range after the migration will result in no communication because the inbound traffic is no longer controlled by Imperva's Web Application Firewall.

For more information on Web Application Firewalls, please visit https://www.f5.com/services/resources/glossary/web-application-firewall
Sep 7, 16:02 CDT
Investigating - In an effort to continuously improve our service levels, TCP has made the decision to migrate to Cloudflare for WAF (Web Application Firewall) services.

TCP will begin routing all traffic through Cloudflare on September 23rd, 2021 at 9:00 PM CST.

We do not anticipate any service interruptions. However, if your organization uses network security rules to control traffic to and from TCP, please make all necessary adjustments to inbound and outbound security rules before September 23rd, 2021 at 9:00 PM CST.

The Cloudflare IP range list can be found here: https://www.cloudflare.com/ips/

Additional Information:

Cloudflare uses an anycast network. This means that domains can potentially resolve to any of the IP addresses listed on the IP range list posted above.

If your organization uses network security rules to control traffic to and from TCP, it is highly recommended that this IP range be whitelisted on your network to prevent any service disruptions.

TCP will continue to send out email reminders regarding the Cloudflare WAF migrations, and any relevant updates regarding the migration will be posted to this status page.
Aug 30, 07:30 CDT
High Memory Utilization
Update - Thank you for your continued patience as we address the performance issues.

We are continuing to investigate several environmental settings to the .NET Core Runtime to increase the activity of the garbage collection behavior and release a significant memory footprint.

We have deployed the release candidate that includes NHibernate fixes to the following server stacks:
• Group2.tcplusondemand.com
• Group5.tcplusondemand.com
• Prod01.tcplusondemand.com
• Prod05.tcplusondemand.com
• Prod02.tcplusondemand.com
• Prod03.tcplusondemand.com
• Prod10.tcplusondemand.com
• Prod11.tcplusondemand.com
• Prod12.tcplusondemand.com
• Prod14.tcplusondemand.com
• Prod15.tcplusondemand.com
• Prod18.tcplusondemand.com
• Prod23.tcplusondemand.com
• Prod25.tcplusondemand.com
• Prod31.tcplusondemand.com
• Prod32.tcplusondemand.com
• Prod34.tcplusondemand.com
• Prod35.tcplusondemand.com

Based on monitoring and performance of the targeted stacks, we are confident that this release candidate will significantly improve performance and efficiency of the TimeClock Plus application.

The Cloud Operations team is implementing new instrumentation to allow for much higher quality monitoring and visibility into the environment health. The tool has been deployed across the entire server fleet, and we are analyzing results and metrics to further improve the performance of TCP software.

We will continue scaling back additional resources allocated to servers as this release candidate is deployed. We are still on track to have it deployed across the entire server fleet by 9/2.
Aug 30, 07:11 CDT
Update - Thank you for your continued patience as we address the performance issues.

We are continuing to investigate several environmental settings to the .NET Core Runtime to increase the activity of the garbage collection behavior and release a significant memory footprint.

We have deployed the release candidate that includes NHibernate fixes to two additional targeted stacks.

Based on monitoring and performance of the targeted stacks, we are confident that this release candidate will significantly improve performance and efficiency of the TimeClock Plus application.

The Cloud Operations team is implementing new instrumentation to allow for much higher quality monitoring and visibility into the environment health. As of today, this tool has been deployed across the entire server fleet.

We will begin scaling back additional resources allocated to servers as this release candidate is deployed. We expect to have it deployed across the entire server fleet by 9/2.
Aug 20, 12:59 CDT
Investigating - Thank you for your continued patience as we address the performance issues.

We are continuing to investigate several environmental settings to the .NET Core Runtime to increase the activity of the garbage collection behavior and release a significant memory footprint.

Targeted hot fixes continue to be deployed weekly to isolated groups of servers and the performance is being monitored for improvements. We have deployed the release candidate that includes NHibernate fixes to a targeted stack and we are monitoring performance.

We anticipate deploying the release candidate that includes NHibernate fixes to two additional targeted stacks on 8/12/2021 to continue testing and monitoring performance.

The Cloud Operations team is implementing new instrumentation to allow for much higher quality monitoring and visibility into the environment health. This tool is going to provide insights to help identify additional areas of improvement. This tool will be fully implemented once 7.1.13.3 has been deployed to all server stacks.

In addition to implementing new instrumentation for higher quality monitoring, the Cloud Operations team is testing and reviewing strategies to increase available resources for servers experiencing increased RAM utilization. The Cloud Operations team began deploying additional auto scaling rules to targeted server stacks this week. We are currently monitoring performance and plan to deploy the new auto scaling rules to all server stacks in the coming weeks.
Aug 4, 06:19 CDT

About This Site

For more information regarding which components effect the availability of your TimeClock Plus v7 On Demand website, please contact our support department at support@timeclockplus.com.

Application Health Operational
Group1.tcplusondemand.com Operational
Group2.tcplusondemand.com Operational
Group3.tcplusondemand.com Operational
Group4.tcplusondemand.com Operational
Group5.tcplusondemand.com Operational
Group10.tcplusondemand.com Operational
Group10-2.tcplusondemand.com Operational
Group12.tcplusondemand.com Operational
Prod01.tcplusondemand.com Operational
Prod02.tcplusondemand.com Operational
Prod03.tcplusondemand.com Operational
Prod04.tcplusondemand.com Operational
Prod05.tcplusondemand.com Operational
Prod06.tcplusondemand.com Operational
Prod07.tcplusondemand.com Operational
Prod08.tcplusondemand.com Operational
Prod09.tcplusondemand.com Operational
Prod10.tcplusondemand.com Operational
Prod11.tcplusondemand.com Operational
Prod12.tcplusondemand.com Operational
Prod13.tcplusondemand.com Operational
Prod14.tcplusondemand.com Operational
Prod15.tcplusondemand.com Operational
Prod16.tcplusondemand.com Operational
Prod17.tcplusondemand.com Operational
Prod18.tcplusondemand.com Operational
Prod19.tcplusondemand.com Operational
Prod20.tcplusondemand.com Operational
Prod21.tcplusondemand.com Operational
Prod22.tcplusondemand.com Operational
Prod23.tcplusondemand.com Operational
Prod24.tcplusondemand.com Operational
Prod25.tcplusondemand.com Operational
Prod26.tcplusondemand.com Operational
Prod27.tcplusondemand.com Operational
Prod28.tcplusondemand.com Operational
Prod29.tcplusondemand.com Operational
Prod30.tcplusondemand.com Operational
Prod31.tcplusondemand.com Operational
Prod32.tcplusondemand.com Operational
Prod33.tcplusondemand.com Operational
Prod34.tcplusondemand.com Operational
phr1.tcplusondemand.com Operational
phr2.tcplusondemand.com Operational
phr3.tcplusondemand.com Operational
phr4.tcplusondemand.com Operational
phr5.tcplusondemand.com Operational
phr6.tcplusondemand.com Operational
phr7.tcplusondemand.com Operational
Prod36.tcplusondemand.com Operational
Prod35.tcplusondemand.com Operational
Prod37.tcplusondemand.com Operational
AWS Infrastructure ? Operational
AWS ec2-us-east-1 Operational
AWS ec2-us-east-2 Operational
AWS rds-us-east-1 Operational
AWS rds-us-east-2 Operational
AWS s3-sa-east-1 Operational
AWS route53-us-east-1 Operational
AWS elb-us-east-1 Operational
AWS elb-us-east-2 Operational
AWS s3-us-east-2 Operational
Web Application Firewall ? Operational
Incapsula Ashburn, VA Operational
Incapsula Dallas, TX Operational
Incapsula Los Angeles, CA Operational
Incapsula Newark, NJ Operational
Incapsula Miami, FL Operational
Incapsula Atlanta, GA Operational
Incapsula Chicago, IL Operational
Incapsula Seattle, WA Operational
Incapsula San Jose, CA Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Past Incidents
Sep 22, 2021

No incidents reported today.

Sep 21, 2021

No incidents reported.

Sep 20, 2021

No incidents reported.

Sep 19, 2021

No incidents reported.

Sep 18, 2021

No incidents reported.

Sep 17, 2021

No incidents reported.

Sep 16, 2021

No incidents reported.

Sep 15, 2021

No incidents reported.

Sep 14, 2021

No incidents reported.

Sep 13, 2021

No incidents reported.

Sep 12, 2021

No incidents reported.

Sep 11, 2021

No incidents reported.

Sep 10, 2021

No incidents reported.

Sep 9, 2021

No incidents reported.

Sep 8, 2021

No incidents reported.

← Incident History Powered by Atlassian Statuspage