ServiceNow ordered a year’s worth of hardware to avoid supply-chain hassles – The Register

The tech world's pandemic supply chain meltdown drove ServiceNow to place orders for a year worth of datacenter kit in January 2022, believing that doing so was necessary to get the hardware it needed to cope with growing customer workloads.

"Pre-COVID, I could generally get stuff in 45 days," CTO Pat Casey told The Register at ServiceNow's Knowledge 22 conference in Sydney, Australia, today.

Well-publicized coronavirus-related supply challenges caused ServiceNow's lead time for some networking kit to stretch to 160 days, while servers can take 120 days to arrive.

So the company "literally placed our entire 2022 order in January," he explained.

"We did it to get in line with the supply chain. If we order it now, hardware starts landing in Q3. If I order in Q3 2022 to meet hardware demand for Q4 2022, I will get the product in Q3 2023."

ServiceNow can't afford to wait that long because, the biz hosts clients on its own infrastructure Casey finds it cheaper to do so. Startups and small companies, he said, rightly balk at paying for datacenter engineers to run their own operations. ServiceNow has reached a scale at which it can afford an infrastructure staff to manage the 200,000 or so instances it runs.

Casey also feels that Amazon Web Services offers "a generic cloud." ServiceNow prefers hardware tuned to the needs of its application, which requires servers loaded up with memory and disk.

"We run one app. I can buy gear optimized for that, which means I can stack it denser; I can often get exactly the stuff I need. The price points are there," Casey elaborated.

The CTO said ServiceNow has found that "middling CPUs" meet its needs. "There is a spectrum of chips: the fast chips with a small number of cores, and the slower you get the more cores they give you for the same amount of power. We are somewhere in the middle we can't run all that efficiently on the core-happy but fairly slow stuff, but it is not worth us to pay a pile of money for something with only four cores on it to get 12 percent faster."

ServiceNow is an x86 shop, though Casey said the company has considered alternatives including IBM's Power architecture. It has not been convinced to change.

ServiceNow's servers use locally attached NVMe storage, housed on separate cards. Shared storage is used sparingly, and usually in the same rack as the servers it, well, serves.

Casey said ServiceNow was one of the first customers of Fusion-io, the storage upstart that was early to market with flash storage on PCIe cards. "It was life changing for us because it was so much better than the spinning disk arrays we had," Casey enthused. "At one point we were buying ten percent of Fusion-io's annual production. We were their number one customer. We are still a big buyer of NVMe storage."

Yet Casey still sees some hangovers from the days of mechanical hard disks in the world of software.

"A lot of the internals of a database are really designed to work around the behaviors of spinning disk arrays," he told The Register. "On NVMe it is almost not worth it. The double write buffering behaviors you see in a lot of databases, you don't need that on NVMe. They are actually counterproductive."

Casey said ServiceNow is a big user of, and investor in, MariaDB, with around 200,000 instances running. In August 2021, ServiceNow acquired German database vendor Swarm64 in the expectation its Postgres-based tech will enable rapid analytics and potentially be useful for primary storage, too. MonetDB, an open source effort led by folks in the Netherlands, also has a home at ServiceNow. Casey said it is "very, very fast, but also sort of fragile."

"You set it up, you load your column storage fast as a thief, but if you change it, it degrades," he said. "So we have to run two of them in parallel, then swap them."

Casey doesnt see any technology on the horizon that he thinks will have a positive impact to compare with that of NVMe, though he is keeping an eye on Compute Express Link (CXL) without being close to a decision. He's aware of SmartNICs but has no plan to adopt them.

One project that has commenced is the development of a backup tier. ServiceNow wrote its own backup solution and is thinking of using a standardized protocol namely, Amazon Web Services' S3 for that tech.

That's not an indication ServiceNow will adopt AWS storage. Instead, Casey said the company may deploy software that speaks the S3 protocol. ServiceNow would not be alone in doing so AWS's cloud storage offering is so pervasive that many on-prem storage rigs and applications use its protocol to allow easier access to hybrid cloud storage. For ServiceNow, S3-compatible storage in-house therefore has value.

Casey's job isn't all infrastructure he also guides development of ServiceNow's products. In that role, he said, an ongoing challenge is the user interface both to ensure complexity does not become an issue and because end-user demands remain high. People expect the ease of consumer tech experiences replicated at work.

More here:

ServiceNow ordered a year's worth of hardware to avoid supply-chain hassles - The Register