Big Data Payday: Rosenblatt Visualizes Success With a Proprietary Platform
Monday, May 07, 2012
WALL STREET & TECHNOLOGY
ANNE RAWLAND GABRIEL
If your firm is still struggling with how to handle today's data deluge, it's not alone. "Although the business intelligence tools market exceeded $120 billion last year, our surveys show only 30 percent of potential users actually use them," notes Rita Sallam, research VP with Gartner Gartner. The challenge, she says, is "having self-service BI tools appropriate for business users. A big reason for lack of adoption is that the tools are just too hard to use."
But an evolving category of data discovery solutions may change all that. "Not only are there dramatic improvements to usability, but also increases in speed, more advanced analytics and gains in the ability for business users to integrate data from different sources," Sallam relates.
This new breed of business user-focused solutions -- offered by both established and emerging vendors -- seeks to improve the ability to access, visualize and make decisions based on larger and more diverse data sets, Sallam explains. "We see this category growing three times faster than the overall market for BI," she says.
Rosenblatt Securities is using just such a solution, a visual analytics platform from Seattle-based Tableau Software, to transform the way it does business. Although the New York-based boutique institutional brokerage and investment banker first adopted Tableau soon after the solution debuted in 2006, Rosenblatt's deployment has since undergone a significant metamorphosis, reports Scott Burrill, partner and managing director at the firm.
"Back then, the crisp PDFs it allowed us to create for client reports caused people to think we had a secret weapon," says Burrill. "But never in my wildest dreams did I think Tableau would give us the ability to explore data the way we do now."
Gaining Momentum During the Downturn
According to Burrill, for Rosenblatt, the big data journey really gained momentum while others were struggling to recover from the 2008 financial crisis. Recognizing the potential of extracting knowledge hidden within growing data volumes, he says, the firm became an early adopter of Seattle-based Amazon's cloud infrastructure. More recently, it moved to Amaxon Elastic Compute Cloud (Amazon EC2). "Using the cloud saved us millions of dollars in capital expenses," Burrill points out.
With storage secured, an extensive evaluation of database technology followed. Finding available solutions cost prohibitive or too inflexible, Burrill adds, Rosenblatt engineered its own database and built a wrapper for it using Redwood Shores, Calif.-based Oracle's MySQL. "Then we started capturing every tick and quote coming off of the U.S. exchanges," he says.
"However, collecting data wasn't the end," Burrill continues. "Having a robust analytical infrastructure with the ability to retrieve, visualize and report that data was every bit as important."
Rosenblatt sought an analytics solution capable of examining hundreds of thousands of trades. But, Burrill notes, most vendors were too "high touch" when it came to requiring end users to "work with them to develop and customize their product," Burrill says. "We wanted tools that gave us instant productivity."
Ultimately, Rosenblatt decided to further leverage Tableau, pioneering new uses for the platform. "Most off-the-shelf solutions don't integrate data sources well," Burrill says.
"The ability to retrieve information in a timely manner is where many solutions fail," he continues. "We were looking for real-time data retrieval, which led to an entirely different framework of storing the data in-memory and computing it in-memory. That's why we chose to create a proprietary system."
On the hardware side, Rosenblatt built several custom analytics servers with massive amounts of graphic processing units (GPU), relative to CPUs, using Nvidia's compute unified device architecture (CUDA). "This gave us lightning-fast, HFT-like computational power," says Burrill. "We're processing data and doing analytics on the entire U.S. market, in some cases in under 4.2 seconds."
5 Doing the Work of 50
Today, Rosenblatt's proprietary solution performs analysis on current-day data and then recursively feeds information back into the firm's master database, which is subsequently used for data discovery, according to Burrill. "With a team of five people we've done with Tableau what it would've taken 50 a significant amount of time to do," he says. "Normally, we may have had to just do sampling. Now we can look at whole populations of data."
Indeed, Rosenblatt performs derived analytics on thousands of different fields. "Then we bring the data into the Tableau dashboard to visualize it, get insights, act on it and tell stories over it -- in a very quick way," Burrill comments.
Of the more than 7,000 stocks Rosenblatt consumes, the firm performs real-time analytics on the entire universe to determine when to enter or exit a position, Burrill adds. "Currently, we're accurate more than 70 percent of the time," he reports.
The daily opportunities provided by the analytics setup, Burrill continues, contribute to Rosenblatt's growth, which includes three new business lines and a 12 percent to 13 percent increase in employee headcount. And going forward, Rosenblatt will roll out proprietary products to its clients based on the firm's big-data system. "We're in alpha on those right now," Burrill relates.
Still, he is quick to point out that "there is no perfect tool, so I'm always exploring. In many instances we'll prototype in Tableau or, lately, [the open source visual programming language] Processing,js."
The bottom line, according to Burrill, is that Rosenblatt's solution keeps it agile and competitive against organizations of all sizes, including those "that out spend us." "We didn't start out thinking we'd use Tableau as a research and development platform," he adds. "But it -- along with the other technologies -- has made us more productive, and that helps make us profitable."