Category Archives: Google

Google Cloud unveiled its "SOC of the Future" today at the Google Cloud Security Summit. (Photo ... [+] Illustration by Pavlo Gonchar/SOPA Images/LightRocket via Getty Images)

Cybersecurity is a top priority for every organization. At least, it should be. The challenge is that the attack surface is expanding, and the threat landscape is adapting so quickly that it is increasingly difficult to keep up with the sheer volume of threatsnever mind effectively defend against them. At the Google Cloud Security Summit today, Google Cloud revealed plans for a SOC of the Future to help companies address these security challenges.

Google Cloud is a leading cloud platform, but it is also increasingly focused on cybersecurity as well. Google Cloud offers Chroniclea cloud service that enables customers to privately retain, analyze, and search massive amounts of security and network telemetry. Googles very DNA is built around the ability to comprehensively index and accurately search essentially all of the information available online. Chronicle takes that model and applies it specifically to security.

Google Cloud has also been busy investing to extend and enhance its security portfolio. Google Cloud partnered with Cybereasonmarrying Chronicle with the Cybereason Defense Platform to deliver Cybereason XDR powered by Google Cloud. The combination of Chronicle to normalize, index, correlate, and analyze data at scale, with the artificial intelligence and MalOp engine of Cybereason yields a powerful tool for defending against attacks.

Chronicle is not the only trick up Google Clouds sleeve, though. Google Cloud also includes VirusTotal, and the recently acquired SOAR (Security Orchestration Automation and Response) solution Siemplify. This arsenal of tools gives Google Cloud a powerful portfolio for cybersecurity.

That is where the SOC of the Future comes in. SOC, for those who dont know, is an acronym for Security Operations Centerthe heart cybersecurity for most organizations.

The core of this initiative is built around Googles Autonomic Security Operations (ASO). ASO is a collection of products, integrations, blueprints, technical content, and an accelerator program to enable customers to take advantage of Googles security operations expertise. The goal is to transform how people approach security challenges, how workflows are engineered to achieve secure outcomes, and how technologies can be leveraged to maximize their value.

I had a chance to speak with Jess Leroy, Google Cloud's Director of Product Management, about the vision for this initiative. Jess explained, The biggest problem in the industry when it comes to SOCs themselves is really that people have been doing things in a way for a long time now that is really no longer sustainable. The old model just doesn't work.

Why? Two reasons. Volume and speed. Jess shared a few statslike a 600% year-over-year growth in crimeware, and the fact that there is over 100 zettabytes of data out there. Meanwhile, threat actors are increasingly using automation to streamline attacks.

Combine that with more pressure for accountability in the C-suite. Jess noted a Gartner statistic that 75% of CEOs will be personally liable for cybersecurity incidents by 2025. He shared that he speaks to CISOs every week and that he sees growing concern over this.

The SOC of the Future will combine the elements of the Google Cloud security portfolio to help security teams evolve from the traditional SOC model to more modern and more agile security operations. Google Clouds goal is to enable more transparent collaboration between service providers and end customersand ensure every role receives relevant data to ensure fast response.

Part of that is based on telemetry. Jess said that most organizations are only using 30% or 40% of their telemetry. They are not correlating and analyzing all of the databecause they simply dont have a framework capable of doing it. They are making educated guesses based on partial snapshots. Chronicle allows Google Cloud to analyze 100% of the available telemetry.

Siemplify is another key element. The SOAR platform provides the ability to act on the analysis. Customized playbooks automate triage and response, which is crucial for keeping up with the volume of threats organizations face.

The SOC of the Future will do for the current SOC what the DevOps revolution did for the NOC [Network Operations Center], declared Sam Curry, CSO of Cybereason. The focus will be on the mission, and the metrics will reflect thatclosing pathways early and often, getting ahead of bad guys, and getting predictive.

Curry added that with this approach, tactical decisions will be based on risk, and that, The SOC of the Future will be extremely efficient in the use of peoples timewhich is the most valuable commodity.

Svetla Yankova, Google Cloud's Global Head of Customer Experience Engineering, walked me through a demonstration of the offering to show me how it analyzes and triages threats. Not only does it use 100% of the telemetry, but once a threat is detected it also has the ability to go back in time to identify and triage other potential instances of the same threat.

Of course, you can only go back as far as the data youve retained, but Google Cloud retains all of your data for a year by default. Hopefully 12 months is far enough to go back to find the first instance of a threat. If not, you probably have bigger issues.

The rest is here:

Google Cloud Launches SOC Of The Future - Forbes

Tim Cook, Apples chief executive, has a longstanding philosophy that Apple should continue to invest for the future amid a downturn. It more than doubled its staff during the Great Recession and nearly tripled its sales. Lately, it has increased bonuses to some hardware engineers by as much as $200,000, according to Bloomberg.

John Chambers, who steered Cisco Systems through multiple downturns as its former chief executive, said the companies strong businesses and deep pockets could afford them the chance to take risks that would be impractical for smaller competitors. During the 2008 downturn, he said Cisco allowed distressed automakers to pay for technology services with credit at a time when competitors demanded cash. The company risked having to write down $1 billion in inventory, but emerged from the recession as the dominant provider to a healthy auto industry, he said.

Companies break away during downturns, Mr. Chambers said.

Excelling will require disregarding the broader markets gloom, said David Yoffie, a professor at Harvard Business School. He said previous downturns had shown that even the strongest businesses were susceptible to profit pressures and prone to pulling back. Firms get pessimistic like everyone else, he said.

The first test for the biggest companies in tech will be contagion from their peers. Amazons shares in the electric vehicle maker Rivian Automotive have plunged more than 65 percent, a $7.6 billion paper loss. Apples services sales are likely to be crimped by a slowdown in advertising by app developers, which rely on venture-capital funding to finance their marketing, analysts say. And start-ups are scrutinizing their spending on cloud services, which will likely slow growth for Microsoft Azure and Google Cloud, analysts and cloud executives said.

People are trying to figure out how to spend smartly, said Sam Ramji, the chief strategy officer at DataStax, a data management company.

Regulatory challenges on the horizon could darken the big tech companies prospects, as well. Europes Digital Markets Act, which is expected to become law soon, is designed to increase the openness of tech platforms. Among other things, it could scuttle the estimated $19 billion that Apple collects from Alphabet to make Google the default search engine on iPhones, a change that Bernstein estimates could erase as much as 3 percent of Apples pretax profit.

But the companies are expected to challenge the law in court, potentially tying up the legislation for years. The probability it gets bogged down leaves analysts sticking to their consensus: Big Tech is going to be more powerful. And whats being done about it? Nothing, Mr. Kramer of Arete Research said.

Jason Karaian contributed reporting.

See the original post:

Big Tech Is Getting Clobbered on Wall Street. Its a Good Time for Them. - The New York Times

NSO Group and its powerful Pegasus malware have dominated the debate over commercial spyware vendors who sell their hacking tools to governments, but researchers and tech companies are increasingly sounding the alarm about activity in the wider surveillance-for-hire industry. As part of this effort, Google's Threat Analysis Group is publishing details on Thursday of three campaigns that used the popular Predator spyware, developed by the North Macedonian firm Cytrox, to target Android users.

In line with findings on Cytrox published in December by researchers at University of Torontos Citizen Lab, TAG saw evidence that state-sponsored actors who bought the Android exploits were located in Egypt, Armenia, Greece, Madagascar, Cte dIvoire, Serbia, Spain, and Indonesia. And there may have been other customers. The hacking tools took advantage of five previously unknown Android vulnerabilities, as well as known flaws that had fixes available but that victims hadnt patched.

It's important to shine some light on the surveillance vendor ecosystem and how these exploits are being sold, says Google TAG director Shane Huntley. We want to reduce the ability of both the vendors and the governments and other actors who buy their products to throw around these dangerous zero-days without any cost. If theres no regulation and no downside to using these capabilities, then youll see it more and more.

The commercial spyware industry has given governments that dont have the funds or expertise to develop their own hacking tools access to an expansive array of products and surveillance services. This allows repressive regimes and law enforcement more broadly to acquire tools that enable them to surveil dissidents, human rights activists, journalists, political opponents, and regular citizens. And while a lot of attention has been focused on spyware that targets Apples iOS, Android is the dominant operating system worldwide and has been facing similar exploitation attempts.

We just want to protect users and find this activity as quickly as possible, Huntley says. We dont think we can find everything all the time, but we can slow these actors down.

TAG says it currently tracks more than 30 surveillance-for-hire vendors that have ranging levels of public presence and offer an array of exploits and surveillance tools. In the three Predator campaigns TAG examined, attackers sent Android users one-time links over email that looked like they had been shortened with a standard URL shortener. The attacks were targeted, focusing on just a few dozen potential victims. If a target clicked on the malicious link, it took them to a malicious page that automatically began deploying the exploits before quickly redirecting them to a legitimate website. On that malicious page, attackers deployed Alien, Android malware designed to load Cytrox's full spyware tool, Predator.

As is the case with iOS, such attacks on Android require exploiting a series of operating system vulnerabilities in sequence. By deploying fixes, operating system makers can break these attack chains, sending spyware vendors back to the drawing board to develop new or modified exploits. But while this makes it more difficult for attackers, the commercial spyware industry has still been able to flourish.

We cant lose sight of the fact that NSO Group or any one of these vendors is just one piece of a broader ecosystem, says John Scott-Railton, a senior researcher at Citizen Lab. We need collaboration between platforms so that enforcement actions and mitigations cover the full scope of what these commercial players are doing and make it harder for them to continue.

Go here to see the original:

Google TAG: Cytrox's Predator Spyware Used to Target Android Users - WIRED

Google has released its annual diversity report, and the company says it made some positive progress over the past year. The company saw its largest increases in representation of Black and Latinx Googlers in the US ever at 20 percent and 8 percent respectively year over year, according to chief diversity officer Melonie Parker. Google also reported improved leadership representation of Black, Latinx and Native American employees by 27 percent, Parker says.

But some data shows there is still more work to be done. The companys US workforce is 33.5 percent women and 66.5 percent men, numbers that are only slightly different than the 32.2 percent women and 67.8 percent men reported in 2021. And in its 2022 report, Google said 48.3 percent of its US workers are white, while 43.2 percent are Asian, 6.9 percent are Latinx, 5.3 percent are Black, and 0.8 percent are Native American.

The company does provide a vast amount of information about its statistics, and you can dig into all of it on the 2022 Diversity Annual Report website and in the PDF version of the report. And Google appears to be committed to moving the needle, saying that as we continue to build a more inclusive and representative Google, well hold ourselves accountable in how we work to make our goals a reality.

Google came under scrutiny for its diversity policies after it fired prominent AI ethicist Timnit Gebru, who is Black, in December 2020. The company made changes to its diversity and research policies two months later.

View original post here:

Google reports increased Black and Latinx representation in the US - The Verge

CEO of Alphabet and Google Sundar Pichai

Mateusz Wlodarczyk | Nurphoto | Getty Images

A new bipartisan proposal takes aim at Google and would force it to break up its digital advertising business if passed.

The Competition and Transparency in Digital Advertising Act was introduced Thursday by a group of key senators on the Judiciary subcommittee on antitrust: the ranking member and chair, Sens. Mike Lee, R-Utah, and Amy Klobuchar, D-Minn., as well as Sens. Ted Cruz, R-Texas, and Richard Blumenthal, D-Conn.

Advertising is a huge part of parent company Alphabet's business. In Q1, Alphabet reported $68.01 billion in revenue, $54.66 billion of which was generated by advertising up from $44.68 billion the year prior.

The bill would ban companies that process more than $20 billion annually in digital ad transactions from participating in more than one part of the digital ad process, according to The Wall Street Journal, which first reported the news.

Google infamously has a hand in multiple steps of the digital ad process, a business that has become the focus of a state-led antitrust lawsuit against the company. Google runs an auction, or exchange, where ad transactions are made and also runs tools to help companies sell and buy ads. If the new legislation passed, it would have to choose in which part of the business it would want to remain.

"When you have Google simultaneously serving as a seller and a buyer and running an exchange, that gives them an unfair, undue advantage in the marketplace, one that doesn't necessarily reflect the value they are providing," Lee told the Journal in an interview. "When a company can wear all these hats simultaneously, it can engage in conduct that harms everyone."

"Advertising tools from Google and many competitors help American websites and apps fund their content, help businesses grow, and help protect users from privacy risks and misleading ads," a Google spokesperson said in a statement. "Breaking those tools would hurt publishers and advertisers, lower ad quality, and create new privacy risks. And, at a time of heightened inflation, it would handicap small businesses looking for easy and effective ways to grow online. The real issue is low-quality data brokers who threaten Americans' privacy and flood them with spammy ads. In short, this is the wrong bill, at the wrong time, aimed at the wrong target."

The coalition behind the bill underscores the way support for reining in tech power through antitrust reform cuts across ideological lines. It's also notable that Lee, the top Republican on the subcommittee, led the bill, given he has opposed some of the other antitrust reforms on the table from Klobuchar and others.

Klobuchar, as chair of the subcommittee, has led an effort to get competition reforms passed this year. So far, two major bills have stood out as having a fighting chance of becoming law if Congress moves on them in time: the American Innovation and Choice Online Act, which would prevent dominant platforms from favoring their own products over those of competitors that rely on their services, and the Open App Markets Act, which would have a similar impact but focuses on app stores like those from Apple and Google. Lee supported the latter, but not the former, during committee votes.

Subscribe to CNBC on YouTube.

WATCH: Here's why some experts are calling for a breakup of Big Tech after the House antitrust report

Continued here:

New bipartisan bill would force Google to break up its ad business - CNBC

Google's eye-catching, 1.1 million-square-foot campus on the edge of the bay is finally complete and open to employees, marking a major expansion of the tech company's office footprint on the Peninsula.

The Bay View campus, located on NASA Ames property off of Moffett Boulevard, will be able to house roughly 4,000 employees across two large and uniquely designed office buildings. The sloped, pavilion-like roofing is clad with large, prismatic glass shingles that generate solar power for the facility, layered in a pattern that the company calls a "dragonscale" solar skin.

Inside, the offices are filled with natural lighting and vibrant colors, with lighthearted-themed areas, or "districts," such as the "Turkey Terrace" and the "Campfire Corner," the latter with meeting rooms modeled to look like tents. The so-called Plankton Palace has a light display shimmering up a stalk of faux seaweed.

The Bay View campus has been in the works for close to a decade, with plans solidifying in 2017 following a long-term lease with NASA Ames. Google's subsidiary, Planetary Ventures, inked a contract with the agency to lease 42 acres for $3.65 million in annual rent.

The company broke ground in 2017, and as of last week began inviting employees to trickle in albeit with plenty of construction workers still working on the finishing touches.

In a tour of the campus Monday, Dave Radcliffe, Google's vice president of real estate and workplace services, said this is a big moment for the company. Most of Google's offices have been inherited or leased, but Bay View marks a rare opportunity to build something new and purposefully built to support the company's workforce.

"This is the first time ... we've been able to move Googlers into a building that we were able to design from the ground up, and so it's an exciting time for us," Radcliffe said.

Even before COVID-19 abruptly forced tech companies in the Bay Area to work from home, the design for Bay View took into account telecommuting and the reality that Google's leadership, candidly, cannot precisely know what the workplace will look like in 20 or 50 years, Radcliffe said. Instead, the company sought to create a flexible workplace environment that can transform numerous times in order to meet new demands.

The first floor, with its art exhibits, food services, couches and giant camel statue, is meant to be an "activation space," Radcliffe said. It's where employees can come in a few times a week for "intentional collaboration" with colleagues, the kind of thing that can be tricky to do from home. The second level is closer to a traditional office setting where Googlers can get work done in a distraction-free environment, but even then there are no defined walls, cubicles or corner offices.

The hybrid work schedule means Bay View needs to accommodate team-focused work, said Michelle Kaufmann, Google's director of real estate and workplace services, research and development. The floor plan, desks and furniture are all movable and interchangeable, with a mix of both group desks and quiet enclosed booths.

"Rather than have the traditional office, which was fixed desks and closed meeting rooms kind of a one-size-fits-all for everyone this new workplace is really about having a much wider range of space types for the activities people are going to be coming back to do," Kaufmann said.

While Bay View's sloped roofs are designed to accommodate 90,000 solar panels and capture energy at all hours of the day, the high steel canopy is also angled to let in plenty of daylight without causing any glare. The opera hall-shaped ceiling, along with acoustic decking and soft materials throughout the building, strike a balance that keeps the office from sounding too loud or too quiet, Kaufmann said.

Google is also touting the new campus' environmentally friendly design, including the largest geothermal pile system in North America. The system involves pipes that run 80 feet underground that can be used to transfer excess heat from the building, which can then be stored for future use during the winter, said Asim Tahir, director of energy and carbon for the company's real estate and workplace services.

The geothermal system means Google will save about 5 million gallons of water each year, and can completely ditch natural gas for heating the building, which is all-electric. The plan is to fully eliminate Bay View's carbon footprint.

"This building will operate with carbon-free energy 90% of the time, and we're hoping through operational improvements we'll over the years get to the full 100%," Tahir said.

A large portion of the leased 42-acre property remains open space, including meadows and marshes, creating a buffer between the newly opened tech campus and the bay coastline. North of the office buildings is a large parking garage next door to multiple buildings with a total of 220 suites similar to motel rooms, built for people visiting Google who need to stay for extended periods.

Bay View is one of multiple Google office developments underway in and around Mountain View. The company's 595,000-square-foot Charleston East campus is still under construction and is expected to finish next year, and the Mountain View City Council approved a third office proposal, the Google Landings project, in 2020. Charleston East is expected to house up to 2,700 Google employees.

Continued here:

Google opens the doors on its massive Bay View campus next to NASA Ames - Palo Alto Online

Indian users of Google Messages are facing a deluge of ads within Androids default messaging app, 9to5Google reports. Users are reporting being sent multiple ads a day, even if they enable the apps Spam Protection feature, and blocking and reporting the offending accounts doesnt appear to prevent ads from different companies appearing. Many of the worst offenders appear to be ads for personal loans, gambling companies, and life insurance.

As 9to5Google notes, companies appear to be abusing an RCS feature designed to let businesses contact their customers with rich, interactive media. Examples listed on Googles site include sending customers QR-code tickets, or letting them make online orders. Instead, some accounts that Google identifies as Verified Business are sending users in India a barrage of ads.

Its unclear how widespread the issue currently is, but Indian-based Ishan Agarwal whose recent tweets have drawn attention to the issue tells The Verge that for him the problem has been growing worse over the course of 2022, and that he receives an average of between two and five ads a day now. Android Police reports that the problem has been around for about a year, and complaints appear to be coming overwhelmingly from users based in India.

Although blocking and reporting accounts prevents specific companies from continuing to send you spam, Android Police reports that the only way to stop receiving ads entirely is to disable Messages support for RCS (under chat features in settings). Its a pretty damning measure to have to take considering how hard Google has been pushing RCS as a successor to SMS and MMS.

Google did not respond to The Verges request for comment.

Excerpt from:

Google Messages users in India are reportedly drowning in ads - The Verge

The Department of Homeland Security has paused work on its recently announced Disinformation Governance Board, according to The Washington Post. The board's director, Nina Jankowicz, has also resigned following an unrelenting stream of harassment. The Post first reported Jankowicz's resignation, and Protocol has since confirmed it.

The board's rollout was shoddy even by DHS Sec. Alejandro Mayorkas' own admission. From the outset, DHS revealed next to nothing about the board's goals or its authorities, leading to concerns that this new entity might be surveilling social media and deciding what does and doesn't constitute disinformation. Conservatives also quickly pounced on Jankowicz, accusing her of being a partisan hack. According to the Post, DHS forbid Jankowicz from saying anything publicly in her own defense.

In truth, the goal of the board, Mayorkas explained far too late, was to do the opposite of what it was being accused of. Throughout DHS, agencies are already working on ways to combat misinformation and disinformation. The purpose of the board was to coordinate those efforts and ensure that they weren't crossing lines with regard to free speech and privacy.

But the decision to do that in a public way rather than in a private audit drew undue scrutiny to the effort. As one source familiar with DHS' plans recently told Protocol, Having a very large governance board and a really big, public rollout for it with a very well-known person in this space very publicly leading it, that probably drove their risk up a little more than it needed to."

In a statement, DHS spokesperson Angelo Fernandez said the board has been "grossly and intentionally mischaracterized," and confirmed that the Homeland Security Advisory Council is now leading a review of the board in hopes of answering two questions. "First, how can the Department most effectively and appropriately address disinformation that poses a threat to our country, while protecting free speech, civil rights, civil liberties, and privacy. Second, how can DHS achieve greater transparency across our disinformation-related work and increase trust with the public and other key stakeholders," Fernandez said.

The final recommendations are due within 75 days, during which time, the board's work will be paused.

Among the board's critics were not just the usual suspects in conservative circles, but also platform regulation scholars and legal experts who feared that the well had already been poisoned. Elon Musk and Jeff Bezos also got in on the act, with Bezos tweeting this week that the "newly created Disinformation Board should review" one of President Biden's tweets about taxing the wealthy to fix inflation.

This story has been updated to include comments from DHS.

Here is the original post:

Apple lost talent due to its return-to-office policy. Google is benefitting. - Protocol

Google announced a new initiative Tuesday aimed at securing the open-source software supply chain by curating and distributing a security-vetted collection of open-source packages to Google Cloud customers.

The new service, branded Assured Open Source Software, was introduced in a blog post from the company. In the post, Andy Chang, group product manager for security and privacy at Google Cloud, pointed to some of the challenges of securing open-source software and stressed Googles commitment to open source.

There has been an increasing awareness in the developer community, enterprises, and governments of software supply chain risks, Chang wrote, citing last years major log4j vulnerability as an example. Google continues to be one of the largest maintainers, contributors, and users of open source and is deeply involved in helping make the open source software ecosystem more secure.

Per Googles announcement, the Assured Open Source Software service will extend the benefits of Googles own extensive software auditing experience to Cloud customers. All open-source packages made available through the service are also used internally by Google, the company said, and are regularly scanned and analyzed for vulnerabilities.

Currently, a list of the 550 major open-source libraries being continuously reviewed by Google is available on GitHub. While these libraries can all be downloaded independently of Google, the Assured OSS program will see audited versions distributed through Google Cloud mitigating against incidents where developers intentionally or unintentionally corrupt widely used open-source libraries. At present, this service is in early access mode and is expected to be made available for wider customer testing in Q3 2022.

The announcement from Google comes as part of an industry-wide drive to improve the security of the open-source software supply chain and one that has also been supported by the Biden administration.

In January, a group of some of the nations largest tech companies met with representatives of federal agencies including the Department of Homeland Security and the Cybersecurity and Infrastructure Security Agency to discuss open-source software security in the wake of the log4j bug. Since then, a recent meeting of the companies involved resulted in a pledge of more than $30 million in funding to boost open-source software security.

Besides contributing funding, Google is also putting engineering hours toward keeping the supply chain secure. The company recently announced the formation of an Open Source Maintenance Crew that would work with the maintainers of popular libraries to improve security.

Read the original here:

Google will start distributing a security-vetted collection of open-source software libraries - The Verge

Why does Google need another database, and why in particular does it need to introduce a version of PostgreSQL highly tuned for Googles datacenter-scale disaggregated compute and storage?

It is a good question in the wake of the launch of the AlloyDB relational database last week at the Google I/O 2022 event.

The name Google is practically synonymous with large scale data storage and manipulation in myriad forms. The company created the MapReduce technique for querying unstructured data that inspired Hadoop, the BigTable NoSQL database, the Firestore NoSQL document database, and the Spanner geographically distributed relational SQL database. These tools were used internally at first, and then put on Google Cloud as the Dataproc, Cloud BigTable, and Cloud Spanner services.

Relational databases are back in vogue, due in part by Google showing that a true relational database is can scale with the advent of Spanner. And to try to encourage adoption of Spanner on the cloud, Google last year created a PostgreSQL interface for Spanner that makes it look and feel like that increasingly popular open source database. This is important because PostgreSQL has become the database of choice in the aftermath of Oracle buying Sun Microsystems in early 2010 and taking control of the much more widely used open source MySQL relational database that Sun itself took control of two years earlier.

The reason why Google needs a true version of PostgreSQL running in the cloud is that it needs to help enterprise customers who are stuck on IBM DB2, Oracle, and Microsoft SQL Server relational databases as their back-end datastores for their mission-critical systems of record get off those databases and not only move to a suitable PostgreSQL replacement, but to also make the move from on-premises applications and databases to the cloud.

That is the situation in a nutshell, Andi Gutmans, vice president and general manager of databases at the search engine, ad serving, and cloud computing giant.

Google has been an innovator on data, and we have had to innovate because we have had these billion user businesses, says Gutmans. But our strength has really been in cloud native, very transformative databases. But Google Cloud has accelerated its entrance into mainstream enterprises we have booming businesses in financial services, manufacturing, and healthcare, and we have focused on heritage systems and making sure that lifting and shifting applications into the cloud. Over the past two years, we have focused on supporting MySQL, PostgreSQL, SQL Server, Oracle, and Redis, but the more expensive, legacy, and proprietary relational databases like SQL Server and Oracle have unfriendly licensing models that really force them into one specific cloud. And we continue to get requests to help customers modernize off legacy and proprietary databases to open source.

The AlloyDB service is the forklift that Google created for this lift and shift, and dont expect for Google to open up all of the goodies it has added to PostgreSQL because these are highly tuned for Google own Colossus file system and its physical infrastructure. But, it could happen in the long run, just as Google took its Borg infrastructure and container controller and open sourced a variant of it as Kubernetes.

As we have pointed out before, the database, not the operating system and certainly not the server infrastructure, is arguably the stickiest thing in the datacenter, and companies make database decisions that span one or two decades and sometimes more. So having a ruggedized, scalable PostgreSQL that can span up to 64 vCPUs running on Google Cloud is important, as will be scaling it to 128 vCPUs and more in the coming years, which Gutmans says Google is working on.

But that database stickiness has to do with databases implementing different dialects of the SQL query language, and also having different ways of creating and embedding stored procedures and triggers within those databases. Stored procedures and triggers essentially embed elements of an application within the database rather than outside of it for reuse and performance, but there is no universally accepted and compatible way to implement these functions, and this has created lock in.

That is one of the reasons why Google acquired CompilerWorks last October. CompilerWorks has created a tool called Transpiler, which can be used to convert SQL, stored procedures, and triggers from one database to another. As a case in point, Gutmans says that Transpiler, which is not yet available as a commercial service, can convert about 70 percent of Oracles PL SQL statements to another format, and that Google Cloud is working with one customer that has 4.5 million lines of PL SQL code that it has to deal with. To help with database conversions, Google has tools to do data replication and scheme conversion, and has provided additional funding where they can get human help from systems integrators.

AlloyDB is not so much a distribution of PostgreSQL as it is a storage layer designed to work with Googles compute and storage infrastructure.

And while Google has vast scale for supporting multi-tenant instances of PostgreSQL, you will not that it doesnt have databases that span hundreds or even thousands of threads. IBMs DB2 on Power10 processors, which has 1,920 threads in a 240-core, 16-socket system with SMT8 simultaneous multithreading turned on, can grab any thread that is not being used by AIX or Linux and use it to scale the database, just to give you a sense of what real enterprise scale is for relational databases. But we are confident that is Google needed to create a 2,000-thread implementation of PostgreSQL, it could do it with NUMA clustering across its network and other caching techniques or by installing eight-way X86 servers that would bring 896 threads to bear with 56-core Sapphire Rapids Xeon SPs and 1,204 threads to bear with 64-core Granite Rapids Xeon SPs. (Again, the operating system would eat a bunch of these threads, but certainly not as much as the database could.) The latter approach using NUMA-scaled hardware is certainly easier when it comes to scaling AlloyDB, but it also means adding specialized infrastructure that is really only suitable for databases. And that cuts against the hyperscaler credo of using cheap servers and only a few configurations of them at that to run everything.

So what exactly did Google do to PostgreSQL to create AlloyDB? Google took the PostgreSQL storage engine and built what Gutmans called a cloud native storage fleet that is linked to the main PostgreSQL node, database logging and point in time recovery for the database runs on this distributed storage engine. Google also did a lot of work on the transaction engine at the heart of PostgreSQL and as a result, Google is able to get complete linear scaling up to 64 virtual cores on its Google Cloud infrastructure. Google has also added an ultra fast cache inside of PostgreSQL, and if there is a memory miss in the database, this cache can bring data into memory with microsecond latencies instead of the millisecond latencies that other caches have.

In initial tests running the TPC-C online transaction processing benchmark against AlloyDB, Gutmans says that AlloyDB was 4X faster than open source PostgreSQL and 2X faster than the Aurora relational database (which has a PostgreSQL compatible layer on top) from Amazon Web Services.

And to match the high reliability and availability of those legacy databases such as Oracle, SQL Server, and DB2, Google has a 99.99 percent uptime guarantee on the AlloyDB service, and this uptime importantly includes maintenance of the database. Gutmans says that other online databases only count unscheduled and unplanned downtime in their stats, not planned maintenance time. Finally, AlloyDB has an integrated columnar representation for datasets that is aimed at doing machine learning analysis on operational data stored in the database, and this columnar format can get up to 100X better performance on analytical queries than the open source PostgreSQL.

The PostgreSQL license is very permissive about allowing innovation in the database, and Google does not have to contribute these advances to the community. But that said, Gutmans adds that Google intends to contribute bug fixes and some enhancements it has made to the PostgreSQL community. He was not specific, but stuff that is tied directly to Googles underlying systems like Borg and Colossus are not going to be opened up.

So now Google has three different ways to get PostgreSQL functionality to customers on the Google Cloud. Cloud SQL for PostgreSQL is a managed version of the open source PostgreSQL. AlloyDB is s souped up version of PostgreSQL. And Spanner has a PostgreSQL layer thrown on top but it doesnt have compatibility for stored procedures and triggers because Spanner is a very different animal from a traditional SQL database.

Here is another differentiator. With the AlloyDB service, Google is pricing it based on the amount of compute and storage customers consume, but the IOPS underpinning access to the database are free. Unmetered. Unlike many cloud database services. IOPS gives people agita because it cannot be easily predicted, and it can be upwards of 60 percent of the cost of using a cloud database.

AlloyDB has been in closed preview for six months and is now in public preview. General availability on Google Cloud is expected in the second half of this year.

Which leads us to our final thought. Just how many database management systems and formats does a company need?

We think of ourselves as the pragmatists when it comes to databases, says Gutmans, who is also famous as the co-founder of the PHP programming language and the Zend company that underpins its support. If you look at the purpose built database, there is definitely a benefit, where you can actually optimize the query language and the query execution engine to deliver best in class price and performance for that specific workload. The challenge is, of course, that if you have too many of these, it starts to become cognitive overload for the developers and system managers. And so theres probably a sweet spot in the middle ground between monolithic and multimodal. You dont go multimodal completely because then you lose that benefit around price, performance, use case specific optimization. But if you go too broad with too many databases, it becomes complicated. On the relational side, customers definitely have at least one relational database and in many cases they also are dealing with legacy database. And with those legacy databases, we are definitely seeing more and more interest in standardizing on a great open source relational database. Document databases provide a lot of ease of use, especially under web facing side of applications when you want to do things like customer information and session management with a very loose schemas, to basically have a bag of information about a customer or transaction or song. I am also a big fan of graph databases. Graph is really going through a renaissance because not only is it very valuable in the traditional use cases around fraud detection and recommendation engines and drug discovery and master data management, but with machine learning, people are using graph databases to extract more relationships out of the data, which can then be used to improve inferencing. Beyond that, we have some other database models that, in my opinion, have some level of diminishing returns, like time series or geospatial databases.

PostgreSQL has very good JSON support now, so it can be morphed into a document database, and it is getting geospatial support together, too. There is a reason why Google is backing this database horse, and getting it fit for the race. It seems unlikely that any relational database could have a good graph overlay, or that a graph database could have a good relational overlay, but that latter item is something to think about another day. . . .

Continue reading here:

Google Needs Another Database To Attack Oracle, DB2, And SQL Server Directly - The Next Platform