Image: Detail from the structure of myoglobin, by AzaToth, Public domain, via Wikimedia Commons.

If U.S. engineers built a spaceship with hyperdrive, and a foreign country managed to reverse-engineer it and figure out how it works, who should get the credit? What is the bigger accomplishment: reverse-engineering a futuristic craft, or designing one from scratch?

DeepMind is a leader in artificial intelligence (AI). Its geniuses managed to beat humans at the popular name Go using its AlphaGo algorithm. Its AI systems have now reached 90 percent success at predicting how a protein will fold. Ablog post from DeepMindexplains why this is a big deal:

In his acceptance speech for the 1972 Nobel Prize in Chemistry, Christian Anfinsen famously postulated that,in theory, a proteins amino acid sequence should fully determine its structure. This hypothesis sparked a five decade questto be able tocomputationally predict a proteins 3D structure based solely on its 1D amino acid sequenceas a complementary alternative to these expensive and time consuming experimental methods.A major challenge, however, is that the number of ways a protein could theoretically fold before settling into its final 3D structure is astronomical.In 1969 Cyrus Levinthal noted that itwould take longer than the age of the known universeto enumerate all possible configurations of a typical protein by brute force calculation Levinthal estimated 10^300 possible conformations for a typical protein.Yet in nature,proteins fold spontaneously, some within milliseconds a dichotomy sometimes referred to as Levinthals paradox.[Emphasis added.]

Reverse-engineering a hyperdrive looks simple by comparison. In 1994, a professor started a contest for AI specialists named CASP: Critical Assessment of [protein] Structure Prediction. Every two years, contestants try to predict a proteins fold from its amino acid sequence alone, without knowing the fold in advance. Before now, scores achieved 20 to 40 on the Global Distance Test (GDT), a measure of the distance between predicted amino acid positionsversusactual biological positions. DeepMind achieved an average score of 60 with AlphaFold in 2018. They increased it enormously this year to 92.4. The blog entry pictures how closely the predicted fold matches the actual fold for two cases. They appear to overlap very closely.

This computational work representsa stunning advanceon the protein-folding problem, a 50-year-old grand challenge in biology. It hasoccurred decades before many people in the field would have predicted. It will be exciting to see the many ways in which it will fundamentally change biological research.

Achieving this success drew inspiration from biology, physics, and machine learning, along with leading experts in protein structure. The team constructed a neural network to approach the challenge, solving small clusters of amino acids then using deep learning methods to explore how they might join up. Even so, the CASP contest uses relatively simple proteins called domains. AlphaFold has more trouble figuring out proteins that interact.Nature Newssays,

The network alsostruggles to model individual structures in protein complexes, or groups, wherebyinteractionswith other proteins distort their shapes.

Nevertheless, the success represents a gigantic leap that will change everything, Ewen Callaway writes. In what ways? John Moult, a professor at the University of Maryland who co-founded CASP, explains inThe Scientist,

This will change medicine. It will change research. It will change bioengineering. It will change everything, Andrei Lupas, an evolutionary biologist at the Max Planck Institute for Developmental Biology in Germany who helped judge the contest, tellsNature, adding that AlphaFold took only 30 minutes to produce the structure of a protein his lab had been trying to figure out for 10 years.

Writing inScienceMagazine, Robert F. Service adds,

Knowing those shapes helps researchersdevise drugsthat can lodge in proteins crevices. And being able to synthesize proteins with a desired structure couldspeed development of enzymes to make biofuels and degrade waste plastic.

This is great news, and rightly applauded. But we need to remember that this folding problem that has baffled humans for 50 years is solved rapidly in living cells at every moment. Levinthal noted that proteins routinely fold spontaneously, some within milliseconds inside the cell. A few need help from chaperones to find their native fold, but many go directly from 1D amino acid sequence to 3D functional protein.

Thats not all. The cell has repair enzymes, too, that can dismantle improperly folded proteins and fix them or replace them if they are irreparable. In our hyperdrive spacecraft analogy, it is like having robots able to detect failed components, pull them out, fix them, and re-insert them. How does a cell without eyes and brains do this? Think of the sophistication of algorithms able to perform such operations!

The concept ofSearchbears heavily in intelligent design theory. If the search problem is complicated enough, succeeding requires additional information beyond what blind search can achieve in the time available. Finding a marked atom in a galaxy, for instance, would take far longer than the age of the universe to succeed. William Dembski proved in his bookNo Free Lunchthat no evolutionary algorithm is superior to blind search, unless auxiliary information is provided. The catch-22, though, is that the searcher needs to search through all possible sources of the auxiliary information to know which one is correct. Dembski likened it to finding a buried treasure by digging at random on an island. That kind of blind search is highly unlikely to succeed if the island is big enough. If the searcher is handed a map, he could go directly to the spot with that auxiliary information. All well and good, like in the movieIts a Mad Mad Mad Mad World, where the information provided was accurate from somebody who knew.

Evolutionary algorithms, though, having no foresight, could generate a billion treasure maps, only one of which might be correct. The search problem then switches to finding the correct map out of billions of treasure maps. If a book on the shelf tells you which map is the correct one (more auxiliary information), how does the searcher know? The searcher would have to check a billion books offering random answers with that information, only one of which might be correct. Each added piece of auxiliary information must be checked out by another search. Thats why no evolutionary algorithm is superior to blind search. The only way to speed to the buried treasure is to trust a source that knows and test the information by digging there. That information must come ultimately from a mind someone who knows the right answer.

Returning to the protein folding problem, we have seen that the search space for protein folds is vast beyond comprehension, like an island as big as the universe. Observing cells routinely folding proteins quickly and accurately, one can conclude therefore that a mind was behind the information. That conclusion is certified by watching AI experts using their minds to reverse engineer protein folds. AI is notinventingsequences that will fold; it is trying to figure out how a given sequence will produce an observed functional fold. Inventing a foldde novois the harder problem (Stephen Meyer discusses functional folds inSignature in the Cell, pp. 99ff.)

Does evolution come into the story at all? Some of the workers in the CASP contest are evolutionary biologists. Thinking that some proteins evolved into other proteins through mutation and natural selection, they believe that similar proteins are connected by common ancestry. This belief, they feel, can allow them to evolve new proteins with similar folds. Callaway says, Some applications, such as the evolutionary analysis of proteins, are set to flourish because the tsunami of available genomic data might now be reliably translated into structures. That is intelligent design, however, not Darwinian evolution; take the word evolutionary out of evolutionary analysis, otherwise it is an oxymoron. There are limits to the amount of variation a fold can endure and preserve function. (Douglas Axe discusses these limits inUndeniable, p. 80ff and 180-182. See coverage byEvolution Newshereandhereabout simplistic evolutionary approaches to solving the folding problem, and why they miss the mark.)

If DeepMinds AlphaFold algorithm succeeds in designing new enzymes, it will be through intelligent design, not blind search. It will build on the information in working proteins, extending that knowledge by design. Materialistic evolutionary processes have no such foresight.

In short, DeepMinds achievement is laudable, but the real prize goes to the designer of protein systems: their encoding in DNA, their translation in the ribosome, their spontaneous (sometimes chaperone-assisted) folding, their functions, their interactions, and their repair mechanisms. All those get perfect scores when not harmed by random mutations that degrade information. AI has not even begun to imitate those capabilities. Any higher scores through AI in the future will be attained by intelligent design, not evolution. The news only underscores the superior knowledge built into the molecular basis of life.

Excerpt from:
Protein Folding Breakthrough: Evolution or Design ...

The high volume of applications submitted to a recent Office of the Vice Chancellor for Research and Graduate Education initiative underscores the serious impact that the COVID-19 pandemic had on research at the University of WisconsinMadison last spring.

The OVCRGE received 110 applications for the Pandemic-Affected Research Continuation Initiative and will support 70. Funded projects come from across campus and represent each of the four research divisions.

Last spring, some researchers were faced with spending down their existing funds while the pandemic limited certain on-site research activities. This included face-to-face human subjects research, research travel and most research activities conducted in-person in university research facilities.

The PARCI supports projects that are now facing a shortage of funds to complete those activities, and is helping to replace critical and time-sensitive research supplies and resources lost due to pandemic-related restrictions. The awards vary, up to $50,000.

We heard many stories about how research progress and funding were impacted by the closure of labs, field work suspension and limitations to other research activities, says Steve Ackerman, vice chancellor for research and graduate education. We knew there was a need for this initiative, even as research activity has successfully restarted on campus.

For example, chemistry professor Tina Wangs research efforts, delayed by the pandemic and resulting campus closures, are being supported by PARCI funding. Her lab is working to develop and use new methods for research in chemical biology, exploring the interplay between protein folding and function, and development of robust sensors and gene circuits. Dysfunctional protein folding is a hallmark of a number of diseases, most notably neurodegenerative disorders.

Michael Cahill, professor of comparative biosciences, received funding to support his work with animal models and to continue funding a graduate student. Cahills research focuses on understanding how gene-based alterations identified in schizophrenia, major depressive disorder and autism spectrum disorders influence neuronal morphology and function.

PARCI is also supporting Dan Vimont, professor of atmospheric and oceanic sciences and co-director of the Nelson Institute Center for Climatic Research along with others at the center to help them more fully resume research into climate variability and climate change, interactions between weather and climate, and global and regional impacts of climate change.

While this initiative will help CCR maintain our pursuit of the Wisconsin Idea through world-class research and outreach on the causes and impacts of climate change, it does more, Vimont says. In addition to recognizing the importance of our colleagues for what they do, it also recognizes the importance of who they are: parents, spouses and family members who are also world-class scientists. As we face what we expect will be a challenging time for the university and for research funding, this is welcome help to our center and to our scientists.

Due to COVID-19, the OVCRGE also has extended end dates on other OVCRGE research-related funding affected by the pandemic and considered reallocations from existing budget line items.

COVID-19 also has had an impact personally on researchers, including faculty members, postdocs, technicians and graduate students. It has affected their educational progress, their career development and their work-life balance. For graduate students and early-career scientists, the disruptions have made it increasingly challenging for them to complete necessary research and to advance their careers.

In response, the Graduate School also recently sponsored a program to support PhD and MFA students facing pandemic graduation delays. The Dissertation Completion Emergency Fellowships program provides one-semester fellowships for students whose graduation has been unavoidably delayed by pandemic-related restrictions who cannot be supported through normal program appointments or endowment funds in Spring 2021 but who now expect to graduate by August 2021. Thirty-nine fellowships are being funded through the DCEF program.

See the article here:
Pandemic-related research initiative receives strong campus response - University of Wisconsin-Madison

For a complete list of our obituaries, seehere.

Jeff McKnight, a molecular biologist at the University of Oregon, died in October at the age of 36.

McKnights research focused on chromatin, a complex of DNA and proteins that controls when and how DNA can be accessed for replication and gene expression. He was one of the earliest researchers in the world capable of directly manipulating its structure, stemming back to his postdoctoral work at the Fred Hutchinson Cancer Research Center using the model organism Saccharomyces cerevisiae. When he had started his own lab in 2016, McKnight said at the time that his real dream was to apply his work to the dozens of human diseases that involve some level of chromatin disruption, including Parkinsons, Alzheimers, and Huntingtons.

Prior to his death from lymphoma, McKnight had spent months chronicling his diagnosis and treatment on social media, prompting an outpouring of support from fellow scientists. He had this humility and vulnerability about him that was really endearing, David Garcia, a molecular biologist at the University of Oregon, told The Scientist.

Biologist Lynika Strozier, a researcher at the Field Museum and an instructor at Malcom X College, died June 7 at age 35 due to complications associated with COVID-19.

After being introduced to molecular biology as an undergraduate at Truman College, Strozier developed a passion for using DNA to identify new and sometimes cryptic species. For her thesis work as a masters student at Loyola University, Strozier sequenced DNA from 200 individual birds in Madagascar thought to belong to three species. Instead, she identified several new species that were indistinguishable based only on the birds appearance.

Her steady hand and aptitude in extracting usable genetic material from old samples earned the admiration of her colleagues. Our entire team entrusted Lynika with extracting DNA from old dried plant material of over 15 years and only very little material from which to do so, Matt Von Konrat, the head of botanical collections at the Field Museum, told The Scientist.

VANDERBILT UNIVERSITY MEDICAL CENTER

Nobel laureate and biochemist Stanley Cohen, who led pioneering studies of cell growth factors, died in February. He was 97.

Stans work not only provided key insights into how cells grow, but it led to the development of many drugs that are used to treat cancer, Lawrence Marnett, the dean of basic sciences at Vanderbilt University, where Cohen taught for 40 years, said to The Tennessean.

Cohens work on different types of growth factors alongside biochemist Rita Levi-Montalcini earned them the 1986 Nobel Prize in Physiology or Medicine. Cohen was honored for his discovery of epidermal growth factora protein that stimulates cell growth and differentiation and plays an important role in tumor developmentwhile Levi-Montalcini was acknowledged for first isolating nerve growth factor. Growth factor receptors have since become the target of numerous drugs, such as gefitinib and cetuximab, that slow or prevent the progression of certain cancers.

SAMARA VISE, KOCH INSTITUTE AT MIT

Angelika Amon, a cell biologist at MIT, died on October 29 from ovarian cancer at the age of 53.

Amon dedicated her career to researching the cell cycle and how disruptions to its normal function can lead to cancer.

During her PhD at the University of Vienna and her subsequent postdoc at MITs Koch Institute for Integrative Cancer Research, Amon used model organisms such as yeast and fruit flies to study how certain proteins and enzymes direct cells through mitosis.

Later, Amon turned her focus to the study of aneuploidy, an abnormal number of chromosomes, and chromosome segregation. She found that extra chromosomes disrupt protein folding and cell metabolism, leading to errors in those processes that can drive cancer.

More than anyone else Ive ever met, she was an absolute force of nature, Matthew Vander Heiden, an MIT biologist and close friend of Amon, told The Scientist. She just has this larger than life personalitytheres no other way to put it.

WILL KIRK/JOHNS HOPKINS UNIVERSITY

Computational biologist James Taylor, who developed a widely used bioinformatics platform, died in April. He was 40.

James made huge contributions to open-source, accessibility, and reproducibility, genomicist Andrew Carroll tweeted following his death. Anyone who runs a bioinformatics tool on the cloud does so thanks to Jamess work.

During his PhD at Penn State University, Taylor helped develop the Galaxy Project, a platform that allows researchers to share genomic data without needing to know how to program. He continued refining the platform as he moved from teaching at Emory University to Johns Hopkins University, and since then Galaxy has been used in more than 10,000 publications across disciplines. Prior to his death, Taylor spoke on Twitter of the need to make transparent, reusable and reproducible analysis pipelines to address the current pandemic, by developing resources for best practices in sharing and analyzing data.

ED SOUZA/STANFORD NEWS SERVICE

Sleep scientist William Dement, who described a number of sleep disorders and opened one of the worlds first sleep disorder clinics, died in June. He was 91.

During his graduate studies at the University of Chicago in the 1950s, Dement studied the physiology of REM sleep and its relationship to dreaming. He later joined the faculty at Stanford University, where he taught for 50 years. There, his focus became the study of sleep apnea and the effects of sleep deprivation. In 1970, he launched the Stanford Sleep Medicine Center and is credited with prompting Congress to establish the National Center on Sleep Disorders Research.

There are not a lot of people who can say they saved the lives of hundreds of thousands of people, Emmanuel Mignot, a professor of psychiatry and behavioral sciences at Stanford University, said in an obituary. But just by pushing this field forward, making sleep apnea recognized, as well as sleep disorders and sleep deprivation, Bill did that.

Wendy Havran, an immunologist at the Scripps Research Institute who studied the role of gamma-delta T cells in wound healing, died January 20 at the age of 64.

Havran first became interested in immunology after meeting John Cambier, an immunologist at Duke University, where she completed her undergraduate degree. While she had intended to study medicine, she became enamored of doing research. It just clicked, and there was no going back, she told The Scientist in a 2019 profile. I wanted to understand how the immune system worked.

During her doctorate research at the University of Chicago, Havran used monoclonal antibodies to study CD4 and CD8 surface markers on T cells. Later, as a postdoc at the University of California, Berkeley, Havran began focusing specifically on gamma-delta T cells, which had only just been described. She was able to map their abundance throughout the body, showing for the first time that they were common in the skin and intestines. In her own lab at Scripps, Havran went on to demonstrate the cells ability to heal wounds and suppress tumor growth.

JOHNS HOPKINS BLOOMBERG SCHOOL OF PUBLIC HEALTH

Immunologist and microbiologist Noel Rose, whose early experiments established the concept of autoimmunity, died of a stroke on July 30 at the age of 92.

Before his pioneering work, it was believed that the body was incapable of launching an immune response against itself. But as a young medical student at the University of Buffalo, Rose showed that rabbits injected with their own thyroid-derived antigens mounted an immune response that damaged or destroyed the animals thyroid. Over the next several decades, he would further unravel the causes and mechanisms of autoimmune diseases, publishing more than 880 articles and book chapters.

In every aspect, [Rose] is the father of autoimmunity, George Tsokos, a professor of rheumatology at Harvard Medical School, told The Scientist in a profile of Rose. The man opened a whole chapter in the book of medicine.

LIZA GREEN, HARVARD MEDICAL SCHOOL

Phillip Leder, a molecular geneticist at Harvard Medical School whose research furthered the fields of molecular biology, immunology, and cancer genetics, died in February. He was 85.

Working alongside NIH geneticist Marshall Nirenberg as a postdoc in the 1960s, Leder developed a technique that confirmed, for the first time, that amino acids were encoded by three nucleotides. Speaking in a 2012 interview, he recalled the excitement of those early experiments. I would go to bed thinking about the next days experiments and then jump out of bed in the morning and rush to the laboratory. It was a lot of work, but the intellectual excitement was enormous.

Having revealed the genetic basis of protein coding, Leder next went on to map the first complete sequence of a mammalian gene, develop the first recombinant DNA vector system, and discover a cancer-causing gene that led to the development of the first mouse model of cancer, among other achievements. He established the genetics department at Harvard Medical School in 1981 only a year after joining the faculty and served as chair for 25 years.

Phil Leder was special. Among great scientists, he was special, and among scientists, he was an icon, David Livingston, a geneticist at Harvard, told The Scientist.

University of california, san diego

Molecular virologist Flossie Wong-Staal, a researcher at the University of California, San Diego (UCSD), who first cloned the human immunodeficiency virus (HIV), died in July at age 73 due to complications from pneumonia.

When Wong-Staal first entered the laboratory of fellow virologist Robert Gallo as a postdoc in 1973, scientists were skeptical that retroviruses could cause cancer in humans. Wong-Staals work helped to overturn this dogma after she and her team identified the first human retrovirus (HTLV-1) and showed that it could indeed lead to cancer. Together, she and Gallo published more than 100 papers in 20 years, making Wong-Staal the most-cited woman in science during the 1980s.

As AIDS cases began to spike in the 1980s, Wong-Staal became the first person to clone HIV the retrovirus that causes the diseaseand began studying the functions of its genes, a necessary step towards developing eventual treatments. She left Gallos lab at the National Cancer Institute in 1990 to launch the Center for AIDS Research at UCSD, where she spent the next several decades studying the virus and developing treatments, many of which are still in use today.

Go here to read the rest:
Those We Lost in 2020 - The Scientist

- Researchers will further develop the Site-directed Enzyme Enhancement Therapy (SEE-Tx) technology for the treatment of rare genetic and neurodegenerative diseases

- The collaborative agreement unites resources from the Institute for Research in Biomedicine (IRB)-USI; Neurocentro -Ente Ospedaliero Cantonale (EOC) & GT GAIN Therapeutics, SA

LUGANO, Switzerland, Dec. 15, 2020 (GLOBE NEWSWIRE) -- GT Gain Therapeutics SA (Gain), a subsidiary of Gain Therapeutics, Inc.,a biotechnology company focused on redefining drug discovery by identifying and optimizing allosteric binding sites that have never before been targeted, along with the Institute for Research in Biomedicine (IRB, affiliated to USI Universit della Svizzera Italiana) and the Neurocentro announced today that Innosuisse, the Swiss Innovation Agency, has agreed to support the CHF 1.5M project by funding approximately CHF 850,000 to leverage these world class research organizations and promote continued innovation in the area of CNS diseases. The remaining support will come from Gain to cover the cost of related headcount expenses being dedicated to the project. The award specifically supports further investigation of the mechanisms of action of Gains proprietary STAR small molecule therapeutic candidates on lysosomal dysfunction and prion-like transmission of toxic forms of protein aggregates associated with neurodegenerative diseases.

Being recognized as an Innosuisse funded innovation project reinforces the support for our innovative approach and unites us with scientists and researchers as passionate as we are to discover new therapeutic approaches using our SEE-Tx target identification platform, said Manolo Bellotto, Ph.D., President and General Manager of Gain. The specific know-how in protein quality control by Prof. Molinari at the IRB and the expertise in neurosciences of Dr. Paganetti from Neurocentro will certainly contribute to a further understanding of the mechanism of action of our molecules in rare and genetic diseases, thus accelerating their development towards the clinic.

Dr.Maurizio Molinari, group leader of the Protein Folding and Quality Control research team from the IRB added, We are honored to be collaborating with the Gain team and to evaluate Gains novel therapeutic candidates as we work to advance new, innovative treatment options for rare lysosomal disorders and neurodegenerative diseases for which there are currently few treatment options. We are grateful to the Swiss Innovation Agency for their support and look forward to initiating this critical research program.

About Gain Therapeutics, Inc.

Gain Therapeutics, Inc. is redefining drug discovery with its SEE-Tx target identification platform. By identifying and optimizing allosteric binding sites that have never before been targeted, Gain is unlocking new treatment options for difficult-to-treat disorders characterized by protein misfolding. Gain was originally established in 2017 with the support of its founders and institutional investors such as TiVenture, 3B Future Health Fund (formerly known as Helsinn Investment Fund) and VitaTech. It has been awarded funding support from The Michael J. Fox Foundation for Parkinsons Research (MJFF) and The Silverstein Foundation for Parkinsons with GBA, as well as from the Eurostars-2 joint program with co-funding from the European Union Horizon 2020 research and Innosuisse. In July 2020, Gain Therapeutics, Inc. completed a share exchange with GT Gain Therapeutics SA., a Swiss corporation, whereby GT Gain Therapeutics SA became a wholly owned subsidiary of Gain Therapeutics, Inc. For more information, visit https://www.gaintherapeutics.com/

About the Institute for Research in Biomedicine (IRB)

The Institute for Research in Biomedicine was founded in 2000 with the clear and ambitious goal of advancing the study of human immunology, with particular emphasis on the mechanisms of host defense. The activities of the 13 research groups now extend beyond immunology to include the fields of DNA repair, rare diseases, structural and cell biology. Located in Bellinzona, capital of the Italian-speaking Canton of Ticino, the IRB is an affiliated institute of the USI Faculty of Biomedical Sciences. For more information, visit : http://www.irb.usi.ch

About Neurocentro -Ente Ospedaliero Cantonale (EOC)

The EOC multisite hospital is organized and managed as a modern company at the service of the patient. It has structures with clear segregations of functions and flexible management systems that foster innovation, accountability and simplification.Our approach favors a collegial and participatory management style. General management and hospital directors form the EOC Management Coordination Conference, physicians are directly involved in EOC management through the Clinical Coordination Conference. The other professional categories actively participate in the management of the EOC within inter-hospital groups.For more information, visit http://www.eoc.ch/en/Centri-specialistici/NSI/NSI.html

Forward-Looking Statements

Any statements in this release that are not historical facts may be considered to be forward-looking statements. Forward-looking statements are based on managements current expectations and are subject to risks and uncertainties which may cause results to differ materially and adversely from the statements contained herein. Such statements include, but are not limited to, statements regarding Gain Therapeutics, Inc. (Gain) expected use of the proceeds from the Series B financing round; the market opportunity for Gains product candidates; and the business strategies and development plans of Gain. Some of the potential risks and uncertainties that could cause actual results to differ from those predicted include Gains ability to: make commercially available its products and technologies in a timely manner or at all; enter into other strategic alliances, including arrangements for the development and distribution of its products; obtain intellectual property protection for its assets; accurately estimate its expenses and cash burn and raise additional funds when necessary. Undue reliance should not be placed on forward-looking statements, which speak only as of the date they are made. Except as required by law, Gain does not undertake any obligation to update any forward-looking statements to reflect new information, events or circumstances after the date they are made, or to reflect the occurrence of unanticipated events.

Gain Therapeutics Investor Contact:Daniel FerryLifeSci Advisors+1 617-430-7576daniel@lifesciadvisors.com

Gain Therapeutics Media Contact:Cait Williamson, Ph.D.LifeSci Communications+1 646-751-4366cait@lifescicomms.com

Read more here:
GT Gain Therapeutics SA Announces Funding from the Swiss Innovation Agency Supporting a 3-year Research Collaboration Project with the Institute for...

(Image: Getty)

COVID-19 lockdowns perhaps gave a few of you some time to organize old photos that have been languishing on SD cards or in boxes, but how many of you built an AI-powered searchable archive of family videos from almost 500 hours of footage?

Dale Markowitz, an Applied AI Engineer and Developer Advocate at Google, did just that. The Texas-based Princeton grad took hours of disorganized, miniDV tape footage housed on Google Drive and turned it into an archive "that let me search my family videos by memories, not timestamps," she wrote in a July blog post. It was the ultimate Father's Day gift.

We spoke with Markowitz recently to find out how machine learning helped her get it done, but why AI is only one part of the puzzle when it comes to solving complex problems.

Although this project used a raft of Google tools, which well get to, it was actually not for the day job, but the coolest Father's Day gift, right?[DM] At Google, I spend lots of time trying to think up new use cases for AI and build prototypes focused on the more business-y side. But I always wanted to work on more fun, zany stuff and, with quarantine, I finally had SO MUCH TIME. So, yes, this one was a gift for my dadwho, by the way, is also a huge programmer nerd who works in machine learning.

As your dad works in machine learning, he would totally get what it took to build it out. Let's go "under the hood" with the details.[DM] Sure. So I uploaded all of my dad's videos to a cloud storage bucket and then analyzed them with the Video Intelligence API, which returns JSON. Basically, the API does all the heavy lifting including: detecting scene changes; extracting text and timestamps on screen using computer vision; transcribing audio; tagging objects and scenes in images; and so on.

Because you needed to apply intelligence to what was probably hours of untagged material, right?[DM] Exactly. When my dad recorded on miniDV, the clips werent saved into separate files. They'd all be smashed into one long, three-hour recording, separated by little flashes of black and white. The API was able to pick out where those clip boundaries should have been.

Regarding audio transcription, that must have helped in tagging, categorizing, and identifying what was on all those miniDVs.[DM] Yes, and I found this to be the coolest part of the project, because it let me search for hyper specific things like "Pokemon" or "Gameboy." Also, my dad was a big video narrator, so I could search his commentary for milestones.

As an applied AI engineer, you're experienced in this field, but others using the API won't need to be up on machine learning, right? Essentially, it's not quite, but almost, out-of-the-box in terms of building out the metadata and intelligence?[DM] Confirmed. You dont need any ML expertise to build out this project. Its very developer-friendly. Having said that, there was one more AI part of this project, which was implementing search. I wanted to be able to search through all those transcripts, scene labels, and objects labels, but I didnt want to have to exactly match the words.

Because you needed a proper semantic search layer for this project?[DM] Exactly. I wanted to allow for near-matches and misspellings and even matching synonyms, such as treating the word trash the same as garbage." As you know, in semantic search," you want an algorithm that understands the semantic meaning of what youre saying regardless of the specific words and spellings you use. For that, I used a great Search as a Service tool called Algolia. I uploaded all my records (as JSON) and Algolia provided me with a smart semantic search endpoint to query those records.

Obviously, youve got a corporate account as a Googler to use all these tools. But what would the cost be for a non-Googler to do this? And are you sharing your GitHub codeyour GitHub code so people can replicate this?[DM] Yep, the code is all open source. Though I should add that a lot of these features are available through Google Photos, which works with videos too, apart from the ability to search transcripts. Cost-wise, I analyzed 126GB of video (about 36 hours) and my total cost was $300. I know that seems high, but it turns out the bulk of the cost came from one single type of analysisdetecting on-screen text. Everything else amounted to just $80. As on-screen text was the least interesting attribute I extracted, I recommend leaving that out unless you really need it. Also, the first 1,000 minutes of video falls in the Google Cloud free tier. Besides the ML parts, storing my data in Algolia runs me around $50 a month for around 90,000 JSON objects. But I havent done much optimizing, and they do have a free tier.

Youre the overall host on YouTube for the new series Making with Machine LearningMaking with Machine Learning." Whats up next there in terms of projects?[DM] Machine-generated recipes, automatically dubbed videos, and an AI dash cam. Well, if I can get those things to workI never really know until I start building them. Another thing Ive been fascinated with lately are ways to do machine learning with little or no data, and zero-shot learning. More on that coming soon.

Well look out for those. Now lets do some background on you: What drew you to study computer science and why specifically at Princeton?[DM] I originally decided to go to Princeton because I wanted to be a theoretical physicist, and I really admired Professor Richard Feynman when I was in high school.But back in 2013, when I was a sophomore in college, it really felt like computer science was the place to be: everything was developing so quicklyArduino, AI, brain-computer interfaces. In retrospect, though I didnt know it then, majoring in computer science was a great decision, because theres almost no field, scientific or otherwise, that hasnt benefited from machine learning. In fact, sometimes it seems like some of the most cutting-edge work in biology and neuroscience and physics is coming from ML.

Whats caught your eye recently in terms of ML?[DM] Specifically within the field of biophysics, Id say DeepMinds new protein folding model, AlphaFold 2, is a great example of ML.

You worked as a researcher on brain-machine interfaces to measure sustained attention. Can you give us a brief explanation of what you were doing there?[DM] In that lab, some researchers had discovered they could (roughly) measure attention by having people do an extremely mundane task in an fMRI machine and then analyzing their brain scans. They were actually using deep learning, which was pretty revolutionary in neuroscience at the time. The problem is that fMRI machines are extremely expensive. I was investigating whether you could get similar results using an EEG machine (which is much cheaper), and specifically a portable, wireless EEG (which is much much cheaper). The results were mixed, but I think, since then, portable EEG machines have gotten better at taking clear readings, and I have gotten better at machine learning.

You moved from data science to applied AI and your focus is on how people can apply AI, ML, etc. But do you also interface with the more theoretical AI people at Google too or only tangentially?[DM] There is a pretty tight relationship between Google Cloud and Google Research. The field changes so quickly that there has to be. When a splashy research paper comes out, it takes almost no time before customers start asking how to get it on Google Cloud. One good example is around explainability and responsible AI. Now that machine learning is becoming more accessible, more folks can build their own models. But how do you know those models are accurate? How do you know you can trust them, and that they wont make predictions that are embarrassing or offensive? The answer is closely linked to explainability, our ability to understand why models make the predictions they doi.e. its hard to trust black box models.

Yeah, theres a big push for explainable AIexplainable AI right now.[DM] This is a tough problem, and an active area of research across Google. But weve been working very closely with Google Research to add explainability into our customer-facing products.

At Google I/OGoogle I/O 2019, you focused on democratizing AIallowing developers to use Googles AI tools, like AutoML, and off-the-shelf APIs to create cool stuff. Tell us more about that. [DM] ML has gotten way easier and more accessible for developers over the past five years. And one of the reasons thats so exciting is because more people from different backgrounds start using it and we end up with very creative projects. Sometimes people see a project Ive built and theyll riff on it, which I think is super cool. For example, I built a tennis serve analyzer, and then some folks built a cricket and a badminton version. I saw a yoga pose detector, and someone built an AI Diary using some of the same tech as my video archive analyzer.

Thinking more broadly, it occured to me that many of your AI-powered projects are applications that could help non-neurotypical people to navigate the world. For example, you engineeredengineered an AI Stylist which could illuminate social cues and help people be workplace appropriate or situation appropriate.[DM] Interesting. On one hand, there are definitely great applications of AI for non-neurotypical folks. The most compelling one Ive heard of involves using computer vision to understand facial expressions and emotions. On the flip side, I try to avoid using machine learning in situations where the result of a mistake is catastrophic.

On that note, when I interviewed Dr. Janelle Shaneinterviewed Dr. Janelle Shane, she had some bizarre brownie recipes generated by one of her AIs, because that stuff is harder than most people imagine. For example, AI doesnt have common sense," so you had to build in rules that a human wouldnt need - i.e. I need two shoes, a left one and a right one, but only one shirt or hat." Any wardrobe mishaps with the stylist before it got it right?[DM] Oh yes, 100%. Furthermore, I would say using a combination of ML and human rules is a pretty good design pattern. One mistake I see people make a lot is try to completely, end-to-end solve a problem with AI. Its better to use ML only for the parts of your system that really need it, such as recognizing a clothing item from an image. But then writing simple rules in places where ML isnt necessarysuch as combining clothing items to make an outfit. Human rulesi.e. An outfit contains exactly two shoesare usually easier to understand, debug, and maintain than ML models. One thing that seemed to trip up the stylist app was that I took a bunch of pictures of clothing on mannequins; my vision model was trained on pictures of people, not mannequins.

The vision model which was looking for humans not static clothes horses?[DM] Yup. That really tricked the model. It was convinced the mannequin was a suitcase or something. By the way, I published the code on GitHub if others want to try it out.

At Google I/O, you also talked about the custom sentiment analysis using natural language. Has that been deployed into something cool like a concurrent translator that can detect irony or emotioni.e. good for non-native speakers while on business trips abroadif we ever get to do those again?[DM] Interesting idea. Were still struggling with irony detection in NLP. But can you really blame a computer for not recognizing irony when lots of humans cant, either?

Good point.[DM] I also suspect irony is largely contextuali.e. text paired with an image, or spoken in a particular way, which makes the problem more challenging. Detecting emotion from speech is a cool idea. But Id probably opt not to analyze just the words the person is saying (text sentiment) and focus more on their intonation. Sounds like a neat project. But like many ML problems, the challenge is finding a good training dataset.

True. So, wrapping it up, do you see the AI tools that youre working with now are a way of building a smart layer between IRL and our silicon cousins (embodied/non-bodied AIs)? For example, when I interviewed AI researcher Dr. Justin Liinterviewed AI researcher Dr. Justin Li, we talked about AI being able to anticipate our needs before we know we have them. [DM] In the future, yes, I think humans and AIs will work closely together. But for me whats most compelling are use cases where machine learning models are uniquely well-suited to do something that humans cant do or arent good at. For example, people make really good assistants and companions and teachers, but theyre not very good at processing millions of web pages in seconds or discovering exoplanets or predicting how proteins fold. So its in these applications, I believe, that AI can make the most impact.

Read more from the original source:
How a Google Engineer Used Her AI Smarts to Create the Ultimate Family Archive - PCMag UK