Amazon EMR is an orchestration tool used to create and run an Apache Spark or Apache Hadoop big data cluster at a massive scale on AWS instances. IT teams that want to cut costs on those clusters can do so with another open source project -- Apache Airflow.

Airflow is a big data pipeline that defines and runs jobs. It works on many tools and services, such as Hadoop and Snowflake, a data warehousing service. It also works with AWS products, including Amazon EMR, the Amazon Redshift data warehouse, Amazon S3 object storage and Amazon S3 Glacier, a long-term data archive.

Amazon EMR clusters can rack up significant expenses, especially if the supporting instances are left running while idle. Airflow can start and take down those clusters, which helps control costs and surge capacity.

Airflow, and its companion product Genie -- a job orchestration engine developed by Netflix -- run jobs by bundling JAR files, Python code and configuration data into metadata, which creates a feedback loop to monitor for issues. This process is simpler than using the spark-submit script or Yarn queues in Hadoop, which offer a wide array of configuration options and require an understanding of elements like Yarn, Hadoop's resource manager.

Therefore, while IT teams don't need Airflow specifically -- all the tools it installs are open source -- it might reduce costs if the organization uses Airflow to install and tear down those applications. Otherwise, Amazon EMR users would have to worry about charges for the idle resources, as well as the costs of a big data engineer and the time and effort required to write and debug scripts.

Let's take a closer look at Amazon EMR and Airflow to see if they fit your organization's big data needs.

Figure 1 shows the configuration wizard for Amazon EMR. It installs some of the tools normally used with Spark and Hadoop, such as Yarn, Apache Pig, Apache Mahout (a machine learning tool), Apache Zeppelin and Jupyter.

The name EMR is an amalgamation of Elastic and MapReduce. Elastic refers to the elastic cluster hosted on Amazon EC2. Apache MapReduce is both a programming paradigm and a set of Java SDKs -- in particular, these two Java classes:

These run MapReduce operations and then optionally save the results to an Apache Hadoop Distributed File System.

Amazon EMR supports multiple big data frameworks, including newer options such as Apache Spark, which performs the same tasks as Hadoop but more efficiently.

Mapping, common to most programming languages, means to run some function or collection of data. Reduce means to count, sum or otherwise create a subset of that now reduced data.

To illustrate what this means, the first programming example for MapReduce that most engineers are introduced to is the WordCount program.

The WordCount program performs both mapping and reducing: The map step creates this tuple (wordX, 1), then counts the number of times a designated value appears. So, if a text contains wordX 10 times, then the map step (wordX,10) counts the occurrence of that word.

Figure 2 illustrates the process of the WordCount program. To begin, let's look at these three sentences:

The first step, map, lists the number of times a given word occurs, and the reduce step further simplifies that data until we are left with succinct tuples: (James, 3); (hit, 1); (ball, 2); and (the, 3).

The WordCount program is far from exciting, but it is useful. Hadoop and Spark run these operations on large and messy data sets, such as records from the SAP transactional inventory system. And because Hadoop and Spark can scale without limit, so can this WordCount approach -- meaning it can spread the load across servers. IT professionals can feed this new, reduced data set into a reporting system or a predictive model.

MapReduce and Hadoop are the original use cases for EMR, but they aren't the only ones.

Java code, for example, is notoriously verbose. So, Apache Pig often accompanies EMR deployments, which enables IT pros to use SQL -- which is shorter and simpler to write -- to run MapReduce operations. Apache Hive, a data warehousing software, is similar.

EMR also can host Zeppelin and Jupyter notebooks. These are webpages in which IT teams write code; they support graphics and many programming languages. For example, admins can write Python code to run machine learning models against data stored in Hadoop or Spark.

Airflow is easy to install, but Amazon EMR requires more steps -- which is, itself, one reason to use Airflow. However, AWS makes Amazon EMR cluster creation easier the second time, as it saves a script that runs with the AWS command-line interface.

To install Airflow, source a Python environment -- for example source py372/bin/activate, if using virtualenv -- and then run this Python package:

Next, create a user.

Then start the web server interface, using any available port.

Shown below is an excerpt of an Airflow code example from the Airflow Git repository. It runs Python code on Spark to calculate the number Pi to 10 decimal places. This enables IT admins to package a Python program and run it on a Spark cluster.

In this snippet:

More here:
Use Amazon EMR with Apache Airflow to simplify processes - TechTarget

For over a decade, the OpenStack Foundation oversaw the open-source OpenStack Infrastructure-as-a-Service (IaaS) cloud. Over time, the OpenStack umbrella covered more open-source projects. So, in October 2020, the Foundation transformed into a new organization: the Open Infrastructure Foundation (OIF). Now, it has announced its first board to help direct its members and their cloud-oriented open-source projects into the 2020s and beyond.

While a good deal smaller than the leading corporate open-source organization, The Linux Foundation, OIF has found a nice niche for itself in covering OpenStack-related open-source software. The OIF will continue to oversee OpenStack. But, with its 100-thousand community members, it will also help direct such cloud-friendly projects as Airship, Kata Containers, Magma, OpenInfra Labs, OpenStack, StarlingX, and Zuul.

The new group's platinum members are Ant Group, AT&T, Ericsson, FiberHome, Huawei, Red Hat, Wind River, and Tencent. They are joined by Facebook, which just became a top-level member. Altogether the OIF has more than 60 corporate members.

The bulk of the organizational work will be done under the OIF's 27-member board. Their numbers include Amar Padmanabhan, Facebook software engineer; Xu Wang; Ant Group senior staff engineer; and Daniel Becker, Red Hat's senior director of engineering. Allison Randal, the well-known open-source strategist; and Perl Foundation leader, will serve as the board chair.

After her election, Randal said:

"Open infrastructure promotes 'innovation and choice on the Internet,' as Mozilla is fond of saying. The open-source nature of the projects hosted at the OpenInfra Foundationas well as the projects with which we openly collaborate -- create economic opportunity around the world. It is an important proof point that modern, open-source development can be funded by corporate interests but guided by the technical governance of individual contributors. I'm humbled by those who put their confidence in me to lead the board, and I'm energized by the opportunities before us to help define the next decade of open infrastructure."

Mark Collier, the OIF's COO, added:

"It's exciting to see the open infrastructure movement grow at such a rapid pace, as evidenced by having more platinum members than we've ever had before, more OpenInfra community members encompassing infrastructure experts who operate some of the largest infrastructures in the world, like Ant Group and Facebook, and new open-source software being created like Magma. All of these trends point to the start of a decade of people and companies investing in open infrastructure that's just getting started, and we want to invite everyone to join us as we build open source communities who write software that runs in production."

View post:
The Open Infrastructure Foundation announces its first board - ZDNet

PHOTO:Steve Snodgrass

Microsofts relationship with open source software has been a complex one. While it has spent a lot of energy in the past knocking open source and its uses in the modern enterprise, there is some evidence to suggest that it is coming around to supporting it again, as what it describes as remote first cultures emerged in response to the COVID-19 pandemic.

The most recent manifestation of its changing relationship with open source comes from Sarah Novotny, the open source lead in the Azure office of the CTO. In a blog post about open source, she seems to suggest that open source can offer a lot to digital workplaces that are trying to manage the COVID-19 change.

2020 fundamentally changed how many companies and teams work seemingly overnight, remote-first cultures became the new norm and people had to change how they communicate and collaborate, she wrote. "However, for those of us who have been deeply engaged in open source, remote work has been our norm for many years because open source communities are large, globally distributed, and require effective collaboration from developers around the world."

While it's unclear how far this ethos carries in Microsoft, what is clear is that open source has become more popular since the pandemic pushed remote working to really take off. In GitHubs 2020 State of the Octoverse report, released in December, the popular code sharing site noted that more than 60 million repositories and more than 1.9 billion contributions were added in the past year. The report noted that much of the growth is happening outside of the United States, with 66% of active users now based outside of North America.

These widely distributed developers, the report showed, already had established remote working practices in place before the broader move to remote work. Even here, though, 2020 challenged companies of all sizes to integrate their open source software experiences and development models in new ways, bringing new learnings as a result.

Justin Rod is CTO of Minneapolis-basedPerforce. He believes that Novotnys blog indicates not just a course variation in Microsofts position on open source but represents a major change in direction for the Redmond, Wash.-based company.

Seeing Microsoft pivot on open source and pivot is an understatement, let us not forget the 'Linux is a cancer' days is a testament both to their agility as a company and the strength of the open source movement at large, he said.

From a wider perspective, open source as a methodology has proven itself to be a larger force than any singular company or entity. Even the Linux Foundation must segment and limit the amount of open source projects that it will throw its weight behind, because the galaxy of software available to promote is so overwhelming.

Remote work, and moreover the conditions in which the entire world had to swiftly embrace remote work, has certainly been easier to navigate for companies that understood how to collaborate using the same methods that open source communities have been relying on for decades. These innovations, borne from the necessities derived by distributed teams of passionate contributors working towards a common goal, can be viably repurposed for business.

In fact, he said, the industry has produced the term inner-source to describe business actions that resemble the practices of open source communities in support of their internal and often proprietary goals. I think youll see that a lot of businesses have learned that they can cut costs without sacrificing productivity, but I dont think that this will be a universal lesson. Software engineers have the luxury of building and using materials that are completely digital, and therefore easy to distribute digitally, he said.

Related Article:12 Productivity Tools Baked Into Office 365

If there has been a steady following for open source over the years, the sudden recent growth in interest and even enterprise acceptance of open source can be traced back to the lockdowns that have been imposed over the past year in response to COVID-19. According to Rhys Davies, product manager at UK-based Canonical, the publisher of Ubuntu, remote working has made people more interested in code tinkering, which in turn has led to more online socializing.

More people are signing up to forums or looking for communities surrounding their hobbies and interests and are involving themselves in these communities, he said. For the tinkerers out there, this has led to a boost in traffic and greater visibility of their pages and discussions. The increase in traffic has led to more contributors, more sign ups and more tutorials.

The size of the open source community has expanded because of this and consequently so has involvement and development in open source tools. Remote work has always been the norm for people working on open source projects.

Until recently the idea of working on open source from an office, or for a company, was extremely rare, he said. Remote work leads this wave of the future as more people will find a home in an open source community that fits their new remote working lives.

A key aspect of this is peoples ability to solve problems through open source software and online communities. These communities enable the sharing of support, advice, ideas, and general encouragement. He added that this simplifies the flow of ideas and means people can share their experiences quicker, as more people working on open source code or ideas will result in quicker turn arounds and better solutions.

The more that big business turns to open source and invests in these tools and platforms, the easier and greater the quality of digital making becomes. Because of this, its easier for people to make their own projects and products, and form communities that carry innovative weight.

However, if Microsoft was truly serious about open source, the company would open source the Windows kernel, Serge Huber, CTO & co-founder of Geneva-based Jahia, said They already provide access to the source code for specific customers (i.e. governments), but it would be truly fantastic if they open sourced even parts of the Windows kernel.

But Microsoft is not at issue here, he added. Open source is indeed the wave of the future, but maybe in ways that are a bit different than most think. As computer systems are aging, open source software becomes the only guarantee that systems can be maintained over the long term, long after companies have come and gone.

Looking back, he said, there are huge problems with software written in the 1980s. All proprietary software is a huge problem when it comes to maintenance issues, and the only systems that are truly maintainable and evolving (i.e. Linux) are the ones that either have big financial backings (Microsoft Windows) or open source software. Even SunOS, at the time, was used in lots of mission-critical solutions; now it only exists because it has been open sourced.

Open source is critical for infrastructure software. What has changed and still needs to change is how to run a profitable business while writing or participating in open source projects. Usually, the open core model is a resilient one, although cloud providers can make this a little tricky at times.

He cites as an example Elastic. In the beginning it had an open core model, but due to business issues, lost part of its competitive edge when Amazon started redeveloping the proprietary parts of Elasticsearch. It is very important that companies leveraging open source have battle-tested business models that are constantly evolving.

See the original post:

How Remote Working and Microsoft Have Given Open Source New Life In The Enterprise - CMSWire

image source: pixabay

Linux tends to be a highly reliable and secure system than any other operating systems (OS). Linux and Unix-based OS have fewer security flaws, as the code is reviewed by a huge number of developers constantly. And anyone has access to its source code. So, you wont need any anti-virus software to protect your PC from malware and viruses. The reason its secure because its open-source, which means you can see its source code. As a result, bugs in the Linux OS will fix rapidly compared to other OS.

If you use windows, you need to buy it or use a crack version of it, like a thief. It just doesnt feel good after a certain age when you use something unethically when there is a free better OS is available for you.

The package-manager of Linux is way more robust than any other OS. Installing software in Linux is notably easy compared to Windows. It can enhance the work-flow for programmers remarkably. Most cases, you only need to open up the terminal and write:

Thats because Linux comes with software managers like apt, rpm , dpkg, synapticetc.

In Windows, you need to search for the website where you can find it. Download the .exe file. Then click on the .exe file. Click, click, click, . Finally, most cases, you need to reboot the system to make the software working. Whereas in Linux, this process is way easier and needs no rebooting to make the software working.

A lot of handy useful programming tools comes pre-installed with Linux. For example, grep, wget, cron etc. Linux also comes with native support for SSH, which helps managing servers swiftly.

It also depends on different distributions of Linux. Like in Kali Linux, you get a lot of pre-installed tools for hacking and penetration testing. This helps a beginner to start the learning process right away.

In Linux, users have full control over updating their system. The system updates are immensely faster in Linux. You can update the system at any time or never, thats your own wish. On the other hand, Windows sometimes forces the user to update the system.

At times, you power up your system, and out of nowhere, the system starts updating, when you maybe need to send an urgent mail to someone. Also, the system updates are notoriously slow in Windows.

Windows always collect user data. Almost everything. Even if they give you the option to diagnostics data viewer, but there are a lot of complain that its just a pretence. Linux doesnt have any hide and seeks game, as you can always look into its source code.

When it comes to Linux, you can literally see everything that is transmitting from your system, if youre tech-savvy enough. Even if youre not, there are tens of thousands of eyes who are looking into the source code and looking for flaws that can cause any vulnerability to the system.

By automating tasks, one can work more efficiently, which makes life easier. Bash shell scripting doesnt come native with windows, and you need to install third-party software to make it work in an ugly way. Experienced Linux users dont write one thing twice because Linux lends itself very well for task automation.

Sometimes we do the same thing again and again, which consumes a lot of time. All of these repetitive tasks can be automate with one-liners. Linux terminal is kind of the heart of Linux. There are Bash, SH, Korn, ZSH, Fish shells, use whatever the one you like. The terminals are all about efficiency. No unnecessary movements, browsing through file manager and clicks.

Linux is a portable OS. It supports a wide range of computer architectures. Portability means you can move your code from one system architecture to another easily. When you need to troubleshoot a system in day to day life, having a lightweight system is amazing.

Now, lets say you have a GitHub repo where there is a set of instructions to install all the software you use in just one file, then what you can do is just execute that one file from anywhere to make your system ready to go with all the software you use in an entirely new Linux installed in your system. Perhaps, within 20 minutes, your system will be ready to go as your daily driver.

If you dont like something in Linux, you can remove it or modify it according to your choice. For example, if you dont like GNOME, you can change it to KDE plasma. Else, you can also install a lot of extensions to enhance your desktop environment or even you can make your own custom UI. You wont find these level of freedom in any other OS.

You can even run your Linux without any UI at all. A lot of people/companies use Linux only to maintain servers, so all you need is just a terminal.

Windows-powered systems require higher hardware requirements. As the OS evolve, with the latest Windows systems, your very old hardware supported PC may become obsolete because it wont support in your old hardware. Whereas, with Linux, you can install it on very low-end hardware systems.

Linux powered PCs are way faster than Windows. The main reason for that Linux a lightweight system and Windows is crammed with lots of unnecessary software. A lot of systems running on the background makes your windows PCs sluggish.

Another reason is, file systems are pretty organized in Linux. Files are located in chunks that are closer to each other, which makes read-write operation way faster than Windows.

Thats the reason, most cloud systems run on Linux. Even Microsoft runs Linux to run Azure. xD

You can run Linux from Super Computers to Game Consoles, Smart TVs, Smartwatches, Car infotainment systems, Flight entertainment systems, Self-driving cars, Nuclear Submarines and many more. NASA heavily relies on Linux for data transmission from satellites and telescopes. Read about this more from here.

Oops! something went wrong type of messages are not useful at all. I know that Windows users are tired of this blue screen.

This sort of error message doesnt help in any way. As a user, I want to know whats actually went wrong. If you can know the explicit error message, you can at least make an attempt to solve the problem.

Linux provides precise error logs that tell you what caused the error. By knowing what went wrong precisely gives you an option to search for the solution on the internet.

Continued here:

13 Reasons Why Linux Is Better Than Windows | by Sadman Kabir Soumik | The Startup | Jan, 2021 - Medium

The Apache Foundation (ASF) has announced the open-source project Apache Superset is now a top-level project. Reaching top-level status indicates that the project has graduated from Apaches incubation program and meets specific ASF requirements.

Apache Superset is designed as a modern data exploration and visualization platform that enables users to build and explore dashboards through a no-code visualization builder and SQL editor. It was first developed in 2015 at Airbnb, and joined Apache in 2017.

Its been amazing to be an active part of growing a welcoming, diverse and engaged community over the past five years while following the ASF principles around inclusion, openness and collaboration, said Maxime Beauchemin, vice president of Apache Superset. At the scale and level of diversity that the Superset project has achieved, its critical to have a solid governance model in place like the one prescribed by the ASF.

The latest version of the project, version 1.0, provides an intuitive interface for users to explore and visualize data sets, create interactive baseboards, and model real-time business intelligence insights at scale, according to the foundation. Key features include the ability to integrate with SQL-speaking data sources, modern cloud-native databases, data warehouses, and engines; support for integrating custom visualization; a thin caching layer for charts and dashboards; code-free visualization building; SQL editor and metadata workflow; enterprise authentication and security model; a lightweight semantic layer; and notification alerts and scheduled reports.

Apache Superset 1.0 is a solid, mature, self-standing solution that fully solves business intelligence and data visualization needs for modern data teams, added Beauchemin. Superset not only covers the table stakes, but also offers guarantees, features and a fresh approach that existing BI solutions cant match.

Apache Superset is also used by Airbnb, American Express, Dropbox, Lyft, Netflix, Rakuten Viki, Twittery and Udemy.

Apache Superset helps Airbnb democratize data insights and make data-informed decisions, said Jeff Feng, product lead at Airbnb and member of the Apache Superset Project Management Committee. Superset uniquely connects SQL analysis with data exploration for thousands of our employees each week. It also serves as a flexible and reliable platform for visualizing metrics, helping executives and knowledge workers see and understand data.

See the original post here:

SD Times Open-Source Project of the Week: Apache Superset - SDTimes.com

Products featured here are selected by our partners at StackCommerce.If you buy something through links on our site, Mashable may earn an affiliate commission.Learn how to code in multiple languages.

Image: Pexels

By StackCommerceMashable Shopping2021-01-23 10:00:00 UTC

With literally hundreds of programming languages to choose from, it's hard to decide which one to start with. That's why weve done a bit of the grunt work for you. Here are some of the most in-demand coding languages to learn in 2021, what makes them unique, and how to get started.

Developed by Google in 2007 APIs and web applications, the open-source Go language is simple, agile, able to handle multicore and networked systems and massive codebases, and used often by large IT companies, data scientists, and more.

For a limited time, you can learn Google Go for just $15 (valued at $119) with this training provided by the EDUmobile Academy. Youll get lifetime access to over three hours of content and even earn a certificate of completion to add to your rsum.

This C# programming master class, led by SkillSuccess instructors like John Bura and Huw Collingbourne, will help you create apps and games of your own. Get lifetime access for just $29 (valued at $1,393) for a limited time.

The general-purpose language Kotlin is widely used for building Android apps, web and desktop apps, and server-side app development. Total beginners can start coding with Kotlin for Android with this 22-hour course from Devslopes. Youll work through hands-on projects like building a fully-functional Slack clone and can sign up for only $19 (valued at $199) for a limited time.

The open-source language R has a moderate learning curve and is often used to build applications for processing statistics. It also boasts an active online community of developers, which is definitely a positive thing when youre first starting out.This $29 (valued at $1,200) R programming crash course is packed with 35 hours of training led by Minerva Singh, a Ph.D. graduate from Cambridge University who works as a data scientist. With lifetime access, you can take a deep dive into todays most important data wrangling and visualization concepts.

Created in 1990, PHP is an open-source programming language used to build over 80 percent of websites on the internet, like Facebook and Yahoo. Its considered relatively easy to learn for first-time coders, and this PHP coding bundle can get you started on the right foot. With a years access to four courses from GreyCampus, youll be able to kickstart your web development career. Learn the essentials of PHP and practice object-oriented programming for just $29.99 (valued at $516) for a limited time.

A powerful general-purpose programming language, C++ is often used to develop operating systems, browsers, games, and more. With this C++ online training, which is taught by software trainer John Purcell, you can learn to use the popular programming language, starting with the basics. Get started for just $14.99 (valued at $600) for a limited time.

One of the most popular programming languages today, Python is also one of the easiest for beginners to learn. This Python training packs a dozen courses from multiple coding experts in which youll actually build real-world applications to understand the language. Valued at over $2,000, you can sign up for lifetime access for just $49.99 for a limited time.

There are tons of ways to learn Java in 2021, but this Java bootcamp really stands out. Its just $35.99 (valued at $990) and includes 10 courses and 10 hours worth of content.

This Ruby on Rails bootcamp can help you get started building sophisticated websites and apps for only $35.99 (valued at $1,000) for a limited time. It includes 42 hours of content from instructors like Mashrur Hossain and Tony Staunton.

Owned by MathWorks and released in the mid-'80s, MATLAB is used to build machine learning and deep learning applications. This complete MATLAB course bundle can help you get started. It includes seven courses and lifetime access to over 30 hours of content for just $34.99 (valued at $3,000) for a limited time.

Developed in 2014 by Apple, Swift is used for building Linux and Mac applications. Start coding in SwiftUI with this developer course from developer Stephen DeStefano: It packs 24 hours' worth of content and only costs $14.99 (valued at $199) for a limited time.

Sign up for the Learn to Code 2021 Bundle and get lifetime access to 27 courses and over 270 hours of content on all the leading coding languages. Led by top-rated instructors like Rob Percival and Nick Walter, youll be able to dip your toes into each of the languages and decide which one feels right to you all for just $59.99 (valued at $4,056) for a limited time.

Original post:

Coding courses on sale: How to learn to code in 2021 - Mashable

After being one of the most requested features for pfSense, Netgate has announced that WireGuard support is coming to the popular open source software for firewalls, VPNs and routers.

The development of a kernel-resident implementation of the new VPN protocol for FreeBSD and pfSense has been in the works for over a year. WireGuard was committed to FreeBSD back in November of last year and the protocol can now be previewed in pfSense Community Edition (CE) 2.5.0.

Users looking to take advantage of the speed and privacy benefits offered by WireGuard can download the source code from the pfSense public code repository and binary images are also available for those tracking the 2.5.0 development branch update from the pfSense GUI.

Netgate also announced that pfSense is rebranding pfSense Factory Edition (FE) to pfSense Plus while PfSense CE will remain as it is though. PfSense FE currently ships with official hardware from Netgate/pfSense and is also available in cloud instances.

According to a new blog post from Netgate, while pfSense FE and pfSense CE have been closely related for some time now, the main ways in which they differ are in support for additional hardware platforms (Arm), support for Cloud Service Provider (CSP) platforms (AWS and Azure) and additional configuration wizards.

The first release of pfSense Plus (21.02) will be made available to all Netgate customers and it will come installed on all of the company's appliances and all new CSP partner marketplace instances.

Interested users can check out this FAQ from Netgate to learn more about pfSense Plus.

Via STH

Read the original:

pfSense adds WireGuard VPN to its platform - TechRadar

Decades ago, a software program called Trillian introduced a way for internet users to interact with multiple IM networks, like ICQ, AIM and MSN Messenger, in a single window. Now, Pebble founder and Y Combinator Partner Eric Migicovsky is revisiting this concept, but this time with a focus on centralizing access to modern-day chat applications. Through the newly launched app, Beeper, users can connect with 15 different messaging services, including WhatsApp, Telegram, Signal, Instagram and Twitter DMs, Messenger, Skype, Hangouts and others even, through a few tricks, iMessage.

Migicovsky says he first came up with the idea for a universal chat app while working on the smartwatch pioneer Pebble, before its acquisition by Fitbit.

We really wanted Pebble to be able to send iMessages, but we could never figure out a way to do it because theres no API for iMessage, he explains. But the idea for Beeper came to a head two years ago when he learned about a protocol called Matrix. All of Beeper is built on top of Matrix, which is this open-source federated, encrypted messaging protocol, he says.

Migicovsky describes Matrix as mostly a hacker thing, but believes its starting to take off among developers. Basically, Matrix offers an API that allows developers to connect with other chat networks using a bridge, which relays the messages back and forth from one side to another.

When I learned about that, I was like Hey, we could build Trillion using Matrix,' Migicovsky says.

Image Credits: Beeper

Migicovsky began to work on Beeper as a side project with Tulir Asokan, a Matrix contributor he met in a Matrix chat room.

To make Beeper (previously called Nova) work with all the different chat apps, they had to build these connecting bridges. This code is also open-sourced and available at Gitlab.com/Nova.

We think its really important for people to know what code theyre running so its all open source. People can inspect it, notes Migicovsky.

Because of this, people also dont have to pay Beeper the $10 per month its charging for access to the service. If they know what theyre doing, they can just run the bridges on their own servers, if they choose.

While every messaging platform has its own unique setup in Beeper, making iMessage work was the most complicated. And the workaround here is somewhat involved, to put it mildly.

Beeper actually ships its users an old, jailbroken iPhone (iPhone 4S, because its cheap) to serve as the bridge. The code installed on the iPhone reads and writes to the database file where your iMessages are stored. The iPhone encrypts the messages with your own private key and then sends it over the Beeper network. This means Beeper, the company, cant read your messages, Migicovsky says.

This process allows Android, Windows and Linux users to use iMessage. But its not the only way Beeper can make iMessages work. Mac users with an always-on device can instead choose to install a Beeper Mac app to work as the bridge.

Migicovsky says hes not afraid of any shutdown attempts or litigation by Apple.

What are they going to do?, he asks, rhetorically.

Even if Apple somehow stopped Beeper from providing jailbroken iPhones to users, the company could redirect their customers to acquire their own old iPhones from Craigslist instead. Meanwhile, the software itself is open source and running on an iPhone at the users house so Beeper isnt really hacking into iMessage itself.

I think given the current climate of messaging freedom I think it would be insane for Apple to start picking a fight with their own users, Migicovsky adds. Plus, he notes that the European Commission is working on draft legislation similar to the GDPR that mandates all companies to open up messaging for other platforms.

When that passes, they legally wont be able to block people from doing something like Beeper, Migicovsky notes.

Image Credits: Beeper

Beeper, of course, is not the first or only startup focused on trying to break through the iMessage lockdown. Other apps have tried to do this in the past, like AirMessage or weMessage, for example. They have only seen limited adoption, however. And Beeper is not the only startup to try to centralize chat applications, either Texts.com is developing a similar system.

That said, signups for Beeper were bigger than Migicovsky expected, he says, though declined to share the details. He says Beeper is slowly onboarding users as a result. (For that reason, we have not been able to actually use Beeper. We cant speak to its claims or usability.)

Despite the competition, where Beeper may have an advantage is in understanding what makes for a great user experience. Pebble, after all, sold over 2 million watches.

Today, Beeper promises features like search, snoozing, archiving, and reminders, and works across MacOS, Windows, Linux, iOS and Android.

Longer term, Migicovsky envisions a platform that could do more than just text and share media, stickers and emoji, like other chat apps. Instead, the team is building a platform that would allow people to build more tools and apps on top of Beeper a system sort of like Gmails plugins. For example, there could be tools that would let users schedule calendar events from within their chats. Or perhaps a tool could help you see all the most recent messages youve had with a particular user across different platforms, like Clearbit.

Migicovsky declined also to detail how the work on Beeper is being financed but when asked if Beeper could be the next step for him as in, a new company to work on he replied, possibly.

Im enjoying my time at YC. It is fantastic. I was just inspired by all the companies that I work with to do this. Part of being VC is talking to all these founders who are building cool stuff and launching it. And I got a little bit jealous, he admits.

Visit link:

Pebble founder launches Beeper, a universal chat app that works with iMessage and others - TechCrunch

Every company is now a software company, or so were told, meaning they have to employ designers and developers capable of building websites and apps. In tandem, the much-reported software developer shortage means companies across the spectrum are in a constant battle for top talent. This is opening the doors to more automated tools that democratize some of the processes involved in shipping software, while freeing developers to work on other mission-critical tasks.

Its against this backdrop that Reflect has come to market, serving as an automated, end-to-end testing platform that allows businesses to test web apps from an end users perspective, identifying glitches before they go live. Founded out of Philadelphia in 2019, the Y Combinator (YC) alum today announced a $1.8 million seed round of funding led by Battery Ventures and Craft Ventures, as it looks to take on incumbents with a slightly different proposition.

Similar to others in the space, Reflect hooks into the various elements of a browser so it can capture actions the user is taking, including scrolls, taps, clicks, hovers, field entry, and so on. This can be replicated later as part of an automated test to monitor the new user signup flow for a SaaS app, for example. If the test later throws up an error, perhaps due to a change made to the user interface, the quality assurance (QA) team can be notified instantly with a full video reproducing the bug, along with relevant logs.

Above: Reflect: Viewing a replay

There are a number of notable players in the automated web testing space, including open source testing framework Seleniumand Cypress, which raised a $40 million funding round just last month. And the low-to-no code space has the likes of Testim, which also covers native mobile apps, and GV-backed Mabl, which was launched by two former Googlers back in 2018. But Reflect is setting out to differentiate its offering in a number of ways.

First up, rather than using browser extensions that record actions locally, Reflect records actions via a virtual machine (VM) in its cloud and screen-shares it back to the user through the Reflect web app. This helps eliminate the causes of common recording errors, like cookies, VPNs, or extensions such as ad blockers that may impact the state of the browser.

In short, Reflect standardizes the testing environment and neutralizes potential inferences, all without requiring any installations.

This approach lets us completely control the test environment, which means we more accurately capture each action you take when testing your site, even for complex actions like drag-and-drops or file uploads, Reflect cofounder Todd McNeal told VentureBeat.

Above: Reflect: Recording a test

McNeal said the company has already amassed more than 80 paying customers who subscribe through a SaaS model that starts at free for three users and 30 minutes of execution time per month. Starter, standard, and enterprise plans offer more features and flexibility.

There are potential downsides to handing full control to a third-partys cloud. Many businesses, particularly larger enterprises, would be more comfortable with an on-premises Reflect installation, something that offers them more control, which would be pertinent if Reflect ever went bust. An open source route might also make some sense for this reason, affording companies greater freedom in terms of how they deploy Reflect. But that would come with major trade-offs in terms of Reflects no-code aspirations.

On-premise installation is something we may add in the future. It has come up with larger enterprises, for sure, McNeal said. Were not considering the open source route though our goal, and what we think the market is looking for, is something that hides away the complexities, and we think the best way to do that is via the no-code approach.

Being truly no-code, as McNeal puts it versus low-code, which may require some form of coding expertise to script specific actions could also help it become the go-to tool for non-developers.

It means that you can truly give our product to anyone in the organization it doesnt have to be just developers, McNeal said. Also, since we dont have the crutch of code to fall back to, it ensures that our recorder needs to be accurate in order to allow customers to test these complex actions.

Its worth noting that Reflect also offers an API and direct CI/CD integrations, enabling its customers to integrate Reflect deeper into their DevOps processes and schedule tests after every deployment, for example, or even after every pull request.

The broader no-code movement has emerged as a major trend in recent years, with Gartner predicting in a 2019 report that by 2023 citizen developers within large enterprises will outnumber professional developers by at least 4 times. This shift is evidenced by a flurry of activity across the space over the past year, with the likes of Amazons AWS launching a no-code app development platform called Honeycode, while Google last year snapped up enterprise-focused AppSheet. Earlier this month, no-code development platform Webflow raised $140 million at a $2.1 billion valuation.

Its clear what benefits automated, no-code platforms could bring to smaller businesses, but why would larger enterprises with plenty of resources be drawn to such tools?

It comes down to what we consider the biggest problems with automated end-to-end testing tools today tests take too long to create and theyre too difficult to maintain, McNeal said. At an enterprise, you have the resources to make this work. You can afford to have developers working full-time on this, who have expertise in the tool necessary to build and maintain your own custom test framework and a suite of code-based tests. But if you can get the same result the same peace of mind that your application works with a lot less time and effort, we think thats a pretty compelling value proposition.

Moreover, even the largest companies have to battle to hire and retain their top technical talent and ensure their time is optimized. By going no-code, they can delegate more QA work to less technically skilled personnel.

It lets enterprises take full advantage of testers in their organization that arent developers, McNeal added. Whereas today those testers are doing primarily manual testing, Reflect actually lets a tester with no coding experience build and maintain entire test suites without any developer intervention.

It is still early days for Reflect. Although its showing some promise, it lacks some of the smarts of its rivals, such as AI or machine learning that can adapt and self-improve over time. However, this is on its roadmap.

Our approach thus far has been to really get the underpinnings of the product correct, and thats rooted in accurately capturing and replicating the actions the user takes in the browser, McNeal said. Well be augmenting this with ML in the future.

Continue reading here:

Reflect brings automated no-code web testing to the cloud - VentureBeat

While Android 11 didn't bring many new features to Android phones it seems 2021's version of the Google-built software will, as we've now heard about one great Android 12 feature.

This comes from GizChina, which spotted a new entry to the Android Open Source Project (AOSP) regarding a 'Share Wi-Fi' feature. This apparently shares Wi-Fi passwords with nearby devices of your choosing, even ones in other rooms to you.

Android is open-source software, meaning people can copy it, tweak it, and make it their own if they want - indeed most phone makers do so for their own handsets. New entries to it, and changes to the source code (as this is) are therefore likely to be new Android features.

This new AOSP entry hasn't been 'merged' yet, which basically means it's not official, but since it came from a Google engineer it could get merged soon. Until then we can't say for sure if it's coming to Android 12, but by the sounds of it, we'd sure hope so.

Current builds of Android let you share Wi-Fi via QR codes, so if you're connected to the Wi-Fi you can get your phone to provide a QR code, and the person who needs the Wi-Fi can scan the code to get access to the internet.

The system works, but it can be a bit fiddly, especially for people who don't know how to scan QR codes or don't have a readily-available scan app.

Apple products like those on iOS, iPadOS and macOS have a much better system - if somebody near you starts trying to connect to the internet, a prompt appears on your device letting you share your Wi-Fi information with them. It's particularly handy if you've got multiple devices you're trying to connect to the web.

That's a beloved feature among Apple fans, and it certainly is handy, so a similar feature in Android phones would be appreciated.

We'll have to wait and see if this feature does make its way to Android 12 at the end of the year when that operating system is set to launch, though the roll-out of various betas in the first half of the year should give us some clues.

Follow this link:

Android 12 looks set to borrow one of the best iOS features - Wi-Fi sharing - TechRadar