(via Fotolia.com)
Why was Dr. John Snow designated the "Father of Epidemiology?" His painstaking investigations of the outbreaks of deadly cholera in London in the 1850s led him to conclude that the disease was caused by contaminated water. His meticulous data gathering pinpointed the source at a single water pump.
Not only had no one ever mapped the incidence of death before, but even the concept of the "germ theory" was still discredited. It took almost twenty years for the scientific and medical profession to accept his premise, but since the water pump was disabled, the cholera epidemic ceased. (See map at end of the piece).
Why did a commercial NLP company, John Snow Labs, choose this name? Though not exclusively producing models for healthcare and life sciences, that is a significant part of their business. I've had the chance to speak with them on several occasions, and they are a remarkable organization, in many ways, which I'll explain. But first, let's review how at least some aspects of NLP work.
By now, everyone is familiar with conversational NLP like Siri. For augmented analytics, the conversation may be, "Download the latest pricing analysis to my phone." The critical thing to remember is that the computer does not understand what you are saying, nor does it understand what it is saying. It can process it and answer, but make no mistake; it's all done with math.
Organizations that offer NLP capabilities do not start from scratch. There are open source libraries that can slot in and wrap their software around it, such as Spark NLP from John Snow Labs, for example. Or other open-source Python libraries such as spaCy, textacy, or nltk. Just to be clear, here are the steps an NLP goes through to satisfy your question. It isn't one model - "Parse my sentence." Each step is a different model. I'm oversimplifying here, but to give you a sense of how part of this works, here are the steps to "understand" a sentence:
Consider that John Snow Labs offers a community version (free) of its Spark NLP that supports an astounding 375 languages, some of which have fewer than 10,000 native speakers. The first question is how and the second question is why. The how is pretty complicated, and I'll save that for another article. But it involves training using deep learning techniques, but the why is pretty compelling.
John Snow Labs is a commercial company focused on Life Sciences, Genomics, and Healthcare. Unlike IBM's proclamation ten years ago that Watson would cure cancer (and failed), John Snow Labs set out to use NLP technology to assist practitioners in assembling credible medical records that are, to this day, scattered, siloed, and inconsistent.
Particularly with oncology, this is crucial because cancer treatment is still very complicated, and practitioners need all the data they can get. hen data is cloistered in multiple EMRs, John Snow Labs frees it. But why 375 languages? As David Talby, the company's founder and CTO said to me recently, accuracy in B2C transactions is useful, but it's not a matter of statistics in oncology. Everyone single person is important, whether they're at Mount Sinai Hospital or a Doctors Without Borders camp.
You may wonder, if these models aren't "smart" in any human intelligence fashion, how can you trust them? Alter all, human language is very complex, often ambiguous if not nonsensical. The answer is that a few years ago, the accuracy of NLP models hovered around 50%. Today, Spark NLP achieves better than 95% accuracy in academic peer-reviewed results.
We have lots of problems with "AI" companies, especially those with venture funding, expected to exhibit the growth their investors demand. As a result, ethical considerations about the products they produce take a severe hit. John Snow Labs is not in that category:
Why is it so crucial for John Snow Labs to have these policies and enforce them? The AI industry is riddled with ethical problems. Many companies engage in a sinister practice, "Ethics washing,"fabricating or exaggerating their commitment to equitable AI. It'sinauthentic and distracts from whether or not actual steps are being taken toward building a world where professional standards demand AI that works just as good for women, people of color, or young people as it does for the white men who make up themajority of people making AI systems.
Training in ethics has not been very effective, at least partly because it's been aimed at AI developers and researchers who make important determinations that can harm people. In contrast, they need to know when the technology benefits and harms. It is clear that better testing and engineering practices, grounded in concern for AI's implications, are urgently needed.
However, focusing on engineers without accounting for the broader political economy within which AI is produced and deployed runs the risk of placing responsibility on individual actors within a much larger system, erasing very real power asymmetries. Those at the top of corporate hierarchies have much more power to set direction and shape ethical decision-making than individual researchers and developers. Racism and misogyny are treated as "invisible" symptoms latent in individuals, not as structural problems that manifest in material inequities. These formulations ignore that engineers are often not at the center of the decisions that lead to harm and may not even know about them. For example, some engineers working on Google's Project Maven weren't aware that they were building a military drone surveillance system. Indeed, such obscurity is often by design, with sensitive projects being split into sections, making it impossible for anyone developer or team to understand the ultimate shape of what they are building and where it might be applied.
In January 2021, John Snow Labs released NLU 1.1, which integrates 720+ new models from the latestSpark-NLP 2.7 release. Including state-of-the-art results with Sequence2Sequence transformers on problems like text summarization, question answering, translation between 192+ languages, and extracted Named Entity in various Right to Left written languages like Arabic, Persian, Urdu, Hebrew, and languages that require segmentation like Korean, Japanese, Chinese, and many more in 1 line of code. These new features are possible because of integratingGoogle's T5 models andMicrosoft's Marian models.
NLU 1.1 has over 1,000 pertained models. In addition to this, NLU 1.1 comes with nine new notebooks showcasing training classifiers for various review and sentiment datasets and seven notebooks for the new features and models. You can browse the complete list of models in this release.
I'll sum it up this way. Facebook is the world's largest deliberate purveyor of disinformation. A company with, in my estimation, no soul. John Snow Labs is a small commercial NLP company of roughly 75 employees that provides an open source library with hundreds of pre-trained models, including tools, in contrast to Facebook, for detecting disinformation.
John Snow's original cholera data points map.
More here:
For John Snow Labs, doing good with NLP is in their DNA (and yours) - Diginomica
- Discovering the mysteries of human DNA - Video [Last Updated On: September 7th, 2012] [Originally Added On: September 7th, 2012]
- Scientists go deeper into DNA - Video [Last Updated On: September 7th, 2012] [Originally Added On: September 7th, 2012]
- Instant Egghead - Genes vs. DNA vs. Chromosomes - Video [Last Updated On: September 7th, 2012] [Originally Added On: September 7th, 2012]
- DNA Calls Out Lineup Of Rappers For Future Battles - Video [Last Updated On: September 7th, 2012] [Originally Added On: September 7th, 2012]
- What is DNA? - Video [Last Updated On: September 7th, 2012] [Originally Added On: September 7th, 2012]
- Turn Your DNA Into Fine Art, BMW Zagato Roadster - Video [Last Updated On: September 7th, 2012] [Originally Added On: September 7th, 2012]
- DNA - OFFICIAL URLTV SUMMER MADNESS 2 RECAP! - Video [Last Updated On: September 7th, 2012] [Originally Added On: September 7th, 2012]
- "Binary DNA" - Video [Last Updated On: September 7th, 2012] [Originally Added On: September 7th, 2012]
- 16x9 - DNA Prophecies: Code reveals your future - Video [Last Updated On: September 7th, 2012] [Originally Added On: September 7th, 2012]
- Gilbert Gottfried - Space DNA, Sexy Weight Loss, Badonkadonk Booty - Gilbert Gets It - Video [Last Updated On: September 7th, 2012] [Originally Added On: September 7th, 2012]
- Animated Health Video Production | DNA Services of America - Video [Last Updated On: September 7th, 2012] [Originally Added On: September 7th, 2012]
- Michael Tsarion ~ Mayans ~ 2012 ~ DNA - Video [Last Updated On: September 7th, 2012] [Originally Added On: September 7th, 2012]
- Mini-drones to take your DNA? - Video [Last Updated On: September 7th, 2012] [Originally Added On: September 7th, 2012]
- C2CAM - DNA Research - 07-09-2012 - Coast To Coast AM - Video [Last Updated On: September 7th, 2012] [Originally Added On: September 7th, 2012]
- Inside The DNA Of MDNA - Video [Last Updated On: September 7th, 2012] [Originally Added On: September 7th, 2012]
- KOTD - Rap Battle - DNA vs Eurgh - Video [Last Updated On: September 7th, 2012] [Originally Added On: September 7th, 2012]
- Starchild DNA Showing "Wright" Stuff - Video [Last Updated On: September 7th, 2012] [Originally Added On: September 7th, 2012]
- Chrome Cats - DNA of a Winner(Official Video) - Video [Last Updated On: September 7th, 2012] [Originally Added On: September 7th, 2012]
- DNA leads to arrest in 1980 murder of Oxnard girl [Last Updated On: September 8th, 2012] [Originally Added On: September 8th, 2012]
- 'Junk' DNA: Not So Useless After All [Last Updated On: September 8th, 2012] [Originally Added On: September 8th, 2012]
- Decoding Human DNA [Last Updated On: September 9th, 2012] [Originally Added On: September 9th, 2012]
- Planet of the Apes: What is that big hunk of 'junk' DNA up to ? [Last Updated On: September 10th, 2012] [Originally Added On: September 10th, 2012]
- Genetics Breakthrough Changes Thinking About DNA [Last Updated On: September 11th, 2012] [Originally Added On: September 11th, 2012]
- 'Junk DNA' and the mystery of mankind's missing genes [Last Updated On: September 11th, 2012] [Originally Added On: September 11th, 2012]
- Real-time observation of single DNA molecule repair [Last Updated On: September 12th, 2012] [Originally Added On: September 12th, 2012]
- Court hears DNA findings in child sex case [Last Updated On: September 12th, 2012] [Originally Added On: September 12th, 2012]
- 2012 International Symposium on Human Identification Features Emerging and Best Practice Forensic DNA Techniques ... [Last Updated On: September 12th, 2012] [Originally Added On: September 12th, 2012]
- DNA could help ID a king [Last Updated On: September 13th, 2012] [Originally Added On: September 13th, 2012]
- DNA with a Twist [Last Updated On: September 13th, 2012] [Originally Added On: September 13th, 2012]
- Three reasons to like junk DNA [Last Updated On: September 13th, 2012] [Originally Added On: September 13th, 2012]
- LBNL Seeks Licensees for Highly Specific and Sensitive DNA Extraction Method [Last Updated On: September 13th, 2012] [Originally Added On: September 13th, 2012]
- Under-twisted DNA origami delivers cancer drugs to tumors [Last Updated On: September 13th, 2012] [Originally Added On: September 13th, 2012]
- DNA ‘junk' contains a treasure of information about disease [Last Updated On: September 14th, 2012] [Originally Added On: September 14th, 2012]
- Research: Hopping DNA supercoils [Last Updated On: September 14th, 2012] [Originally Added On: September 14th, 2012]
- DNA evidence missing in Assange case [Last Updated On: September 16th, 2012] [Originally Added On: September 16th, 2012]
- Missing DNA evidence in Assange case [Last Updated On: September 16th, 2012] [Originally Added On: September 16th, 2012]
- No Assange DNA on torn condom - report [Last Updated On: September 16th, 2012] [Originally Added On: September 16th, 2012]
- Calif. DNA Collection From Arrestees Challenged [Last Updated On: September 17th, 2012] [Originally Added On: September 17th, 2012]
- Federal appeals court to hear challenge to California DNA collection law [Last Updated On: September 17th, 2012] [Originally Added On: September 17th, 2012]
- Applied DNA Sciences Contracts With Inventionland [Last Updated On: September 18th, 2012] [Originally Added On: September 18th, 2012]
- Applied DNA Sciences, Textile Centre of Excellence Unveil Textiles Anti-Counterfeiting Platform [Last Updated On: September 18th, 2012] [Originally Added On: September 18th, 2012]
- Rapist caught by DNA test jailed [Last Updated On: September 18th, 2012] [Originally Added On: September 18th, 2012]
- FBI eager to embrace mobile 'Rapid DNA' testing [Last Updated On: September 19th, 2012] [Originally Added On: September 19th, 2012]
- Expansion of criminal DNA collection proposed [Last Updated On: September 19th, 2012] [Originally Added On: September 19th, 2012]
- Assessment of HPV DNA Alone Insufficient to Identify HPV-Driven Head and Neck Cancers [Last Updated On: September 19th, 2012] [Originally Added On: September 19th, 2012]
- George Zimmerman's DNA, not Trayvon Martin's, found on gun [Last Updated On: September 20th, 2012] [Originally Added On: September 20th, 2012]
- George Zimmerman: No DNA evidence of a struggle for his gun [Last Updated On: September 20th, 2012] [Originally Added On: September 20th, 2012]
- DNA evidence links Vallejo man to January stabbing in SLO, police say [Last Updated On: September 20th, 2012] [Originally Added On: September 20th, 2012]
- Legal hurdles threaten to slow FBI's 'Rapid DNA' revolution [Last Updated On: September 21st, 2012] [Originally Added On: September 21st, 2012]
- Judge denies motions to dismiss DNA evidence in Hudson murder case [Last Updated On: September 22nd, 2012] [Originally Added On: September 22nd, 2012]
- Researchers report novel approach for single molecule electronic DNA sequencing [Last Updated On: September 22nd, 2012] [Originally Added On: September 22nd, 2012]
- Novel approach for single molecule electronic DNA sequencing [Last Updated On: September 22nd, 2012] [Originally Added On: September 22nd, 2012]
- DNA helps Wyckoff police nab 'motorcycle burglar' [Last Updated On: September 22nd, 2012] [Originally Added On: September 22nd, 2012]
- Novel DNA barcode engineered: New technology could launch biomedical imaging to next level [Last Updated On: September 25th, 2012] [Originally Added On: September 25th, 2012]
- DNA Microarray 2012: A Focus on Sales Growth [Last Updated On: September 25th, 2012] [Originally Added On: September 25th, 2012]
- DNA in 1980 Maine murder case shown to match defendant [Last Updated On: September 25th, 2012] [Originally Added On: September 25th, 2012]
- DNA recovered during Rayney probe [Last Updated On: September 26th, 2012] [Originally Added On: September 26th, 2012]
- FBI makes headway on DNA testing backlog, report says [Last Updated On: September 26th, 2012] [Originally Added On: September 26th, 2012]
- Male DNA found for first time in female brains [Last Updated On: September 27th, 2012] [Originally Added On: September 27th, 2012]
- Bearing Sons Leaves Male DNA Traces in Mom's Brain [Last Updated On: September 28th, 2012] [Originally Added On: September 28th, 2012]
- Many female brains contain male DNA [Last Updated On: September 28th, 2012] [Originally Added On: September 28th, 2012]
- New drive to take criminals' DNA [Last Updated On: September 28th, 2012] [Originally Added On: September 28th, 2012]
- DNA remains focus in Highway of Tears cases [Last Updated On: September 28th, 2012] [Originally Added On: September 28th, 2012]
- Analysing The Evidence On DNA [Last Updated On: September 29th, 2012] [Originally Added On: September 29th, 2012]
- DNA Clears Death Row Inmate [Last Updated On: September 29th, 2012] [Originally Added On: September 29th, 2012]
- Burn victim identified by DNA in maggots [Last Updated On: September 29th, 2012] [Originally Added On: September 29th, 2012]
- DNA fails to match couple on two other skeletons [Last Updated On: September 29th, 2012] [Originally Added On: September 29th, 2012]
- DNA Dynamics Update on Sports Title [Last Updated On: September 30th, 2012] [Originally Added On: September 30th, 2012]
- DNA solves teen's 1974 murder [Last Updated On: September 30th, 2012] [Originally Added On: September 30th, 2012]
- Some Women's Brains Contain Male DNA: Study [Last Updated On: September 30th, 2012] [Originally Added On: September 30th, 2012]
- DNA exonerates man after 15 years on death row - Video [Last Updated On: September 30th, 2012] [Originally Added On: September 30th, 2012]
- DNA link prompts charges in cold case rapes - Video [Last Updated On: September 30th, 2012] [Originally Added On: September 30th, 2012]
- DNA testing has its limits [Last Updated On: October 1st, 2012] [Originally Added On: October 1st, 2012]
- DNA evidence exonerates 300th prisoner nationwide [Last Updated On: October 1st, 2012] [Originally Added On: October 1st, 2012]
- DNA testing facility in Pune to speed up cases in Mumbai [Last Updated On: October 1st, 2012] [Originally Added On: October 1st, 2012]
- Rape DNA process 'not adequate' [Last Updated On: October 2nd, 2012] [Originally Added On: October 2nd, 2012]
- IntegenX Announces U.S. Launch of the RapidHIT™ 200 System – Rapid DNA Technology That Will Revolutionize the Use of ... [Last Updated On: October 2nd, 2012] [Originally Added On: October 2nd, 2012]
- 300th person exonerated by DNA evidence [Last Updated On: October 2nd, 2012] [Originally Added On: October 2nd, 2012]
- Inherited Diseases Found Sooner in Newborns With DNA Scan [Last Updated On: October 3rd, 2012] [Originally Added On: October 3rd, 2012]
- Woman charged in husband's death gives DNA sample [Last Updated On: October 3rd, 2012] [Originally Added On: October 3rd, 2012]