This content is part of the Essential Guide: How to solve your TMI problem: Data science analytics to the rescue
Manage Learn to apply best practices and optimize your operations.

Why a big data algorithm belongs on your dev agenda

When Liz Rush started her career in software development, working in algorithm programming and the use of algorithms in data analytics was not on her agenda.

“Just the term ‘algorithm development’ frightened me back then, and many developers I meet have the same feeling,” said Rush, developer evangelist for Algorhitmia, an open marketplace which now offers 1,600+ algorithms. Her job is demystifying algorithms, algorithm development and the use of pre-built algorithms to speed up software development projects and enable data analytics.

Liz Rush talks about why a big data algorithm should be on your dev agenda

Liz Rush talks big data and algorithms at the 2016 LWT Summit in San Francisco

I talked with Rush about how shared algorithms eliminate a build-your-own (BYO) bottleneck in the application development process. We also chatted about her presentation, “Your Big Data’s Boring: Let’s Talk about Algorithms,” which she delivered at the 2016 Lesbians Who Tech Summit in San Francisco.

Finding a love for coding

Rush became a software developer through Ada Developers Academy, a non-profit, tuition-free code school for women in Seattle. “That’s when I fell in love with coding,” she said. Before that, she worked in marketing and translation jobs, where she acquired skills that still come into play in her developer evangelist role.

“I do a lot of code writing and producing content, like sample apps,” she said. She also creates demos and presentations that show how to reap the benefits of sharing algorithms between departments within an enterprise.

Why big data algorithms matter

At the LWT Summit, Rush told the audience of 1,500 tech pros how they can use algorithms to get value out of data. “Big data is useless unless you have the tools to do anything with it,” she said. Via algorithms, she explained, developers can see patterns in data of all types, such as images and audio, to classify unstructured data and do other analytics tasks.

Rush also discussed career and community-building opportunities in algorithm development during the LWT Summit. She said the growing algorithm outsourcing movement allows users to pick and choose their algorithms, giving a large number of software teams access to quality algorithms developed by experts.

Until recently, most enterprise development teams built their own algorithms. “Everybody wanted to do it themselves, sometimes just to prove that they could do it themselves,” Rush said. “I understand that impulse, as you can learn a lot from the process.”

But these teams should realize that while a BYO approach to creating algorithms worked pretty well when businesses used fewer applications, today it can limit the ability to rapidly develop new products and services. Applications are evolving rapidly, Rush noted, and shared algorithms help businesses keep their apps on the leading edge.

The value of an algorithm marketplace

Many companies understand they need an algorithm to do a specific task, but acquiring those needed algorithms is not always easy. “Traditionally, they’ve had to take the costly path of building it in-house or hiring an algorithm developer,” Rush said. Via a marketplace, businesses have the less-costly option of choosing pre-designed algorithms rather than reinventing the wheel themselves.

Start-ups, in particular, can benefit for using an algorithm marketplace. “They can get the value from the algorithm quickly without paying to build it or risking building it improperly,” she said. “Sharing algorithmic knowledge is a sure way to improve development practices.”